%g{dZddlZddlZddlmZddlmZmZmZm Z m Z m Z m Z m Z mZddlZddlZ ddlZdZddlmZmZmZmZddlmZmZdd lm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(d d l)m*Z*d d l+m,Z,Gd dZ-y#e$rdZYJwxYw)z> Container for the result of running the sample (MCMC) method N)StringIO) AnyDictHashableListMutableMappingOptionalSequenceTupleUnionTF)_CMDSTAN_SAMPLING _CMDSTAN_THIN_CMDSTAN_WARMUP_TMPDIR)Method SamplerArgs) EXTENSIONbuild_xarray_datacheck_sampler_csv cmdstan_pathcmdstan_version_beforecreate_named_text_file do_commandflatten_chains get_logger)InferenceMetadata)RunSetceZdZdZdeddfdZdefdZdedejfdZ de fd Z e defd Ze deefd Ze defd Ze defd Ze defdZe deedffdZe deefdZe deejfdZe deejfdZe defdZe deejfdZe deejfdZddddededejfdZ de!ee"ffdZ#d-dZ$d-dZ% d.de&ed ede'jPfd!Z)deefd"Z* d/d#e+eeedfdede'jPfd$Z, d/d#e+eeedfdedd%fd&Z- d0d'ededejfd(Z.de!eejffd)Z/de!eejffd*Z0d1d+eeddfd,Z1y)2 CmdStanMCMCa Container for outputs from CmdStan sampler run. Provides methods to summarize and diagnose the model fit and accessor methods to access the entire sample or individual items. Created by :meth:`CmdStanModel.sample` The sample is lazily instantiated on first access of either the resulting sample or the HMC tuning parameters, i.e., the step size and metric. runsetreturnNcH|jtjk(s$tdj |j||_|j j j}t|tsJt|_ |j|j|_ t|_|j|j|_t |_|j$|j$|_|j&|_|j*|_|j j.|_t3j4d|_t3j4d|_t3j4d|_t3j<|j j>t@|_!t3j<|j j>t@|_"|jG}tI||_%|j(s|jMyy)zInitialize object.z=Wrong runset method, expecting sample runset, found method {}Ndtype)'methodrSAMPLE ValueErrorformatr!_args method_args isinstancerr _iter_sampling iter_samplingr _iter_warmup iter_warmupr_thinthin fixed_param_is_fixed_param save_warmup _save_warmupsig_figs _sig_figsnparray_draws_metric _step_sizezeroschainsint _divergences_max_treedepths_validate_csv_filesr _metadata_check_sampler_diagnostics)selfr! sampler_argsconfigs S/var/www/dash_apps/app1/venv/lib/python3.12/site-packages/cmdstanpy/stanfit/mcmc.py__init__zCmdStanMCMC.__init__=s }} -""(&"7  {{((44  +   $5  % % 1"."<"??rLclttj|j|jz S)zn Number of sampling (post-warmup) draws per chain, i.e., thinned sampling iterations. )rArjrkr.r2rds rJnum_draws_samplingzCmdStanMCMC.num_draws_samplings' 499d11TZZ?@AArLc|jS)z Returns object which contains CmdStan configuration as well as information about the names and structure of the inference method and model output variables. )rErds rJmetadatazCmdStanMCMC.metadatas~~rL.c4|jjdS)a9 Names of all outputs from the sampler, comprising sampler parameters and all components of all model parameters, transformed parameters, and quantities of interest. Corresponds to Stan CSV file header row, with names munged to array notation, e.g. `beta[1]` not `beta.1`. column_names)rEcmdstan_configrds rJrrzCmdStanMCMC.column_namess~~,,^<> ( ( 2h > L  J  ||rLcV|j|js |jSdS)z Step size used by sampler for each chain. When sampler algorithm 'fixed_param' is specified, step size is None. N)rbr5r>rds rJ step_sizezCmdStanMCMC.step_sizes' &*&:&:tDDrLc|jS)zF Period between recorded iterations. (Default is 1). )r2rds rJr3zCmdStanMCMC.thins zzrLc6|js |jSdS)z Per-chain total number of post-warmup divergent iterations. When sampler algorithm 'fixed_param' is specified, returns None. N)r5rBrds rJ divergenceszCmdStanMCMC.divergencess )-(<("?@ @{{9:q!+,,rLc i}t|jD]q}|dk(rt|jj||j |j |j|j|j}|j rp|d|j|<|d|j|<t|jj||j |j |j|j|j}|D]L}|dvs||||k7stdj|jj|||||||j rN|d|j|<|d|j|<t|S)a Checks that Stan CSV output files for all chains are consistent and returns dict containing config and column names. Tabulates sampling iters which are divergent or at max treedepth Raises exception when inconsistencies detected. r)pathis_fixed_paramr/r1r6r3ct_divergencesct_max_treedepth) stan_version_majorstan_version_minorstan_version_patch stanc_versionrP num_samples num_warmupr6r3refreshzFCmdStan config mismatch in Stan CSV file {}: arg {} is {}, expected {})ranger@rr!rSr5r.r0r7r2rBrCr)r*)rGdzeroidrestkeys rJrDzCmdStanMCMC._validate_csv_filesst{{#3 HAAv)..q1#'#7#7"&"5"5 $ 1 1 $ 1 1 +++01A+BD%%a(.34F.GD((+)..q1#'#7#7"&"5"5 $ 1 1 $ 1 1 !C  "#J%*4(88> $ 5 5a 8 # %c %c 9%6+++01A+BD%%a(.34F.GD((+g3 Hh rLc tj|js tj|jrdg}|jj d}t |jjD]}|j|dkDr?|jd|dzd|j|d|j||z dzd d |j|dkDsg|jd|dzd|j|d |j||z dzd d |jd tjd j|yy)zR Warn if any iterations ended in divergences or hit maxtreedepth. z(Some chains may have failed to converge.rrzChain rz had z divergent transitions (dz.1fz%)z iterations at max treedepth (zQUse the "diagnose()" method on the CmdStanMCMC object to see further information.rON) r:anyrBrCrErsrr!_chainsappendrrrR)rG diagnosticsct_itersrs rJrFz&CmdStanMCMC._check_sampler_diagnostics\sk 66$## $t/C/C(DEFK~~44]CH4;;../ $$Q'!+&& QuT->->q-A,BC"//28;S@#FbJ ''*Q.&& QuT-A-A!-D,EF"2215h>CSIM    /  L [!9 :))ErLc H |jjdk7ry|j}d}|jr||jz }|j}t j ||jt|jftd|_t j |jt|_ t|jD]}t|jj|d5}|j!j#}t|dkDrO|j%dr>|j!j#}t|dkDr|j%dr>|j&sx|jrtt|jD]\}|j!j#}|j)d }|Dcgc] }t|c}|j||ddf<^|j!j#}|d k7r)|d k7r$|j!j#}|d k7r$|j!j#}|j)d \} } t| j#|j|<|j*j,d d k7r|j!j#}|j!j/dj1}t|j)d } |dk(ri|j2dk(r-t j |j| ft|_n-t j |j| | ft|_|ra|j2dk(r=|j)d }|Dcgc] }t|c}|j4|ddf<n|j#j)d }|Dcgc] }t|c}|j4|dddf<td| D]k}|j!j/dj1}|j)d }|Dcgc] }t|c}|j4||ddf<mnP|j7} |j!j#}|j%ds|j9| t||D]\}|j!j#}|j)d }|Dcgc] }t|c}|j||ddf<^ ddd|jJycc}wcc}wcc}wcc}wcc}w#1swYxYw)z Allocates and populates the step size, metric, and sample arrays by parsing the validated stan_csv files. )rNrF)r&orderr%r#,z# Adaptation terminated=rurxz # diag_er)r<shapernr7rlr:emptyr@lenrrfloatr>ropenr!rSreadlinestriprZr5splitrErslstriprstriprvr=tellseek) rG num_drawssampling_iter_startchainfdlinerxsxrYr{num_unconstrained_paramsposs rJrbzCmdStanMCMC._assemble_drawsvs ;;   $ ++     .. .I"&"7"7 hh  S):):%; <  ((4;;e<4;;'A FEdkk++E2C8@ FB{{}**,$i!m(<;;=..0D$i!m(<++((!&t'<'(J!{{}224!{{}33F;BBD36tzz#3G0 A:#//8;/1xx%)[[2J$K*/0" 02xx(, (@(@%& +0 0"  #//8;%)ZZ_LN9Oq%(9O UAX 6%)ZZ\%7%7%<68="12E!H=" UAq[ 9*/q2J)K!&A+-;;=+?+?+G+N+N+PD)-CB:<A&56aA&DLL1$=!&!ggi!{{}224#s3GGCL2I>FA;;=..0DCBBD/EQa/EDKK5! ,F{@ F@ FA FD{{&&&m8N>:P=" A&0FA@ F@ Fs`7A;V3A$VU> )A"V E/V;V 9VVA&V>V B*V:V V>VV! percentilesr8c t|dk(r tdd}|D]}|dkDs||kDs td|}ddjd|D}t|tr |dks|d kDrtd j ||j xsd }||kDrtjd ||d |}tjjtddtz}dj |jjj} t!t"| dd} dj | } t%ddrdj | } |||| g|jj&z} t)| dt+| d5} t-j.| ddddd}ddd|j0r*j2Dcgc]}|j5d c}n0j2Dcgc]}|dk(xs|j5d c}}d|j2_||S#1swYxYwcc}wcc}w) aN Run cmdstan/bin/stansummary over all output CSV files, assemble summary into DataFrame object. The first row contains statistics for the total joint log probability `lp__`, but is omitted when the Stan model has no parameters. The remaining rows contain summary statistics for all parameters, transformed parameters, and generated quantities variables, in program declaration order. :param percentiles: Ordered non-empty sequence of percentiles to report. Must be integers from (1, 99), inclusive. Defaults to ``(5, 50, 95)`` :param sig_figs: Number of significant figures to report. Must be an integer between 1 and 18. If unspecified, the default precision for the system file I/O is used; the usual value is 6. If precision above 6 is requested, sample must have been produced by CmdStan version 2.25 or later and sampler output precision must equal to or greater than the requested summary precision. :return: pandas.DataFrame rzUInvalid percentiles argument, must be ordered non-empty list from (1, 99), inclusive.czQInvalid percentiles spec, must be ordered non-empty list from (1, 99), inclusive.z--percentiles= rc32K|]}t|ywra)str).0rs rJ z&CmdStanMCMC.summary..s&C!s1v&Csrz@Keyword "sig_figs" must be an integer between 1 and 18, found {}z[Requesting %d significant digits of output, but CSV files only have %d digits of precision.z --sig_figs=bin stansummaryzstansummary-{}-z.csvT)dirprefixsuffix name_onlyz--csv_filename={}z --csv_file={}N)fd_outrbrhigh) delimiterheader index_colcommentfloat_precision__lp__)rr)rRr-rAr*r9rrosrrrr!r+ model_namerrrrSrrpdread_csvr5indexendswithname)rGrr8cur_pctpctpercentiles_str csv_sig_figs sig_figs_strcmd_path tmp_csv_file tmp_csv_pathcsv_strrNr summary_datarmasks rJsummaryzCmdStanMCMC.summaryst4 { q ;  CRxsW} ?G  chh&C{&CCD E (C(HqLHrM"F8, ~~* l " L 5   %XJ/ 77<< NE=9#< )// 0A0A0L0LM - Vt &,,\: !!R (%,,\:G      KK ! ! " 3t$ , % ;; & L ##,8+=+= >aD! ! >>J=O=O89V 31::d#333 #' D!!#   ?sH5I< I5H>ctjjtddtz}|g|j j z}t}t|||jS)a Run cmdstan/bin/diagnose over all output CSV files, return console output. The diagnose utility reads the outputs of all chains and checks for the following potential problems: + Transitions that hit the maximum treedepth + Divergent transitions + Low E-BFMI values (sampler transitions HMC potential energy) + Low effective sample sizes + High R-hat values rdiagnose)rNr) rrrRrrr!rSrrgetvalue)rGrrNresults rJrzCmdStanMCMC.diagnose*sU77<< zI7MNj4;;000s6*  rLvarsc|t|tr|g}n|}|r%|jstj d|j g}|t jD]}||jjvr|j|-||jjvrL|jj|}|j|j|j|j|dvr|j|t!d|ngdt#|jz}|j%|}|j&\}} } t)j*t)j,d| dz|j/d| |j0} t)j2t)j,d|dz| j/d| |j0} t)j,d|| zdzj/d| |j0} t)j4| | | |gd}t7j8t;|gdt#|jz|S) a Returns the sample draws as a pandas DataFrame. Flattens all chains into single column. Container variables (array, vector, matrix) will span multiple columns, one column per element. E.g. variable 'matrix[2,2] foo' spans 4 columns: 'foo[1,1], ... foo[2,2]'. :param vars: optional list of variable names. :param inc_warmup: When ``True`` and the warmup draws are present in the output, i.e., the sampler was run with ``save_warmup=True``, then the warmup draws are included. Default value is ``False``. See Also -------- CmdStanMCMC.draws CmdStanMCMC.draws_xr CmdStanGQ.draws_pd UDraws from warmup iterations not available, must run sampler with "save_warmup=True".)chain__iter__draw__zUnknown variable: rrr)axis)datacolumns)r-rr7rrrbdictfromkeysrE method_varsr stan_varsextendrrrend_idxr)listrrr:repeatarangereshapeTtile concatenater DataFramer)rGrr vars_listcolsvarryrn_drawsn_chainsrY chains_coliter_coldraw_cols rJdraws_pdzCmdStanMCMC.draws_pd>s+0  $$!F  d// L =    }}Y/ A$..444KK$DNN444>>33C8DKK))$..4<<H;;KK$$'9#%?@@ A3T$:K:K5LLD j 1${{1 IIbii8a<0' : WQ' * Q  GGBIIa1-x 8 WQ' * Q  IIa'H,1 2 WQ' * Q   HhFQO||&3d4;L;L6MM   rLz xr.Datasetcts td|r%|jstj d|.t |j jj}nt|tr|g}n|}|j|j}|j j}|dd|dd|d|d|d}|r*|jr||jz }|j|d <i}|jt!j"|d }|D]6} t%||j j| |j'| 8t)j*||| j-d ddS)a Returns the sampler draws as a xarray Dataset. :param vars: optional list of variable names. :param inc_warmup: When ``True`` and the warmup draws are present in the output, i.e., the sampler was run with ``save_warmup=True``, then the warmup draws are included. Default value is ``False``. See Also -------- CmdStanMCMC.draws CmdStanMCMC.draws_pd CmdStanGQ.draws_xr z>Package "xarray" is not installed, cannot produce draws array.rr.rrrP) stan_versionrPrnrl)rdrawr)coordsattrsrr.)XARRAY_INSTALLED RuntimeErrorr7rrrrErkeysr-rrbrnrsrlrhr:rrrxrDataset transpose) rGrrrrmetarr coordinatesr s rJdraws_xrzCmdStanMCMC.draws_xrs$ P  d// L =  <T^^55::<=I c "II ++ ~~,,#$89:!()*!D1E,F+GI']"+ 0  $++ .. .I(,(=(=E$ %.0^^IIi(6  C ((- j 1   zz${%@JJ VS  rLr c |j|d}|jj|j|}|S#t$rDt d|ddj |jjjzwxYw)a Return a numpy.ndarray which contains the set of draws for the named Stan program variable. Flattens the chains, leaving the draws in chain order. The first array dimension, corresponds to number of draws or post-warmup draws in the sample, per argument ``inc_warmup``. The remaining dimensions correspond to the shape of the Stan program variable. Underlyingly draws are in chain order, i.e., for a sample with N chains of M draws each, the first M array elements are from chain 1, the next M are from chain 2, and the last M elements are from chain N. * If the variable is a scalar variable, the return array has shape ( draws * chains, 1). * If the variable is a vector, the return array has shape ( draws * chains, len(vector)) * If the variable is a matrix, the return array has shape ( draws * chains, size(dim 1), size(dim 2) ) * If the variable is an array with N dimensions, the return array has shape ( draws * chains, size(dim 1), ..., size(dim N)) For example, if the Stan program variable ``theta`` is a 3x3 matrix, and the sample consists of 4 chains with 1000 post-warmup draws, this function will return a numpy.ndarray with shape (4000,3,3). This functionaltiy is also available via a shortcut using ``.`` - writing ``fit.a`` is a synonym for ``fit.stan_variable("a")`` :param var: variable name :param inc_warmup: When ``True`` and the warmup draws are present in the output, i.e., the sampler was run with ``save_warmup=True``, then the warmup draws are included. Default value is ``False``. See Also -------- CmdStanMCMC.stan_variables CmdStanMLE.stan_variable CmdStanPathfinder.stan_variable CmdStanVB.stan_variable CmdStanGQ.stan_variable CmdStanLaplace.stan_variable TrzUnknown variable name: z Available variables are z, )rrErextract_reshapeKeyErrorr)rRr)rGr rrouts rJr\zCmdStanMCMC.stan_variables` JJ*DJIE"nn66s;KKCJ )#/++))DNN4499;<=  s (- (-T=T#s(^=~;4T'p&1\"c]\"\"  \"|!(3-!,-1 MDIsD()MM  M`LQ> #tCy$./> DH> > F!< <<  <|S"**_ 5&  $sBJJ"7  '#'$'rLr ).r0rjriortypingrrrrrr r r r numpyr:pandasrxarrayrr ImportError cmdstanpyr rrrcmdstanpy.cmdstan_argsrrcmdstanpy.utilsrrrrrrrrrrprr!rr r$rLrJr=s    QP6   (J 'J '+sA77BB