pyhsmm - library for approximate unsupervised inference in Bayesian Hidden Markov Models (HMMs) and explicit-duration Hidden semi-Markov Models (HSMMs), focusing on the Bayesian Nonparametric extensions, the HDP-HMM and HDP-HSMM, mostly with weak-limit approximations.

Build Status

Bayesian inference in HSMMs and HMMs

This is a Python library for approximate unsupervised inference in Bayesian Hidden Markov Models (HMMs) and explicit-duration Hidden semi-Markov Models (HSMMs), focusing on the Bayesian Nonparametric extensions, the HDP-HMM and HDP-HSMM, mostly with weak-limit approximations.

There are also some extensions:

Installing from PyPI

Give this a shot:

pip install pyhsmm

You may need to install a compiler with -std=c++11 support, like gcc-4.7 or higher.

To install manually from the git repo, you'll need cython. Then try this:

python setup.py install

It might also help to look at the travis file to see how to set up a working install from scratch.

Running

See the examples directory.

For the Python interpreter to be able to import pyhsmm, you'll need it on your Python path. Since the current working directory is usually included in the Python path, you can probably run the examples from the same directory in which you run the git clone with commands like python pyhsmm/examples/hsmm.py. You might also want to add pyhsmm to your global Python path (e.g. by copying it to your site-packages directory).

A Simple Demonstration

Here's how to draw from the HDP-HSMM posterior over HSMMs given a sequence of observations. (The same example, along with the code to generate the synthetic data loaded in this example, can be found in examples/basic.py.)

Let's say we have some 2D data in a data.txt file:

$ head -5 data.txt
-3.711962552600095444e-02 1.456401745267922598e-01
7.553818775915704942e-02 2.457422192223903679e-01
-2.465977987699214502e+00 5.537627981813508793e-01
-7.031638516485749779e-01 1.536468304146855757e-01
-9.224669847039665971e-01 3.680035337673161489e-01

In Python, we can plot the data in a 2D plot, collapsing out the time dimension:

import numpy as np
from matplotlib import pyplot as plt

data = np.loadtxt('data.txt')
plt.plot(data[:,0],data[:,1],'kx')

2D data

We can also make a plot of time versus the first principal component:

from pyhsmm.util.plot import pca_project_data
plt.plot(pca_project_data(data,1))

Data first principal component vs time

To learn an HSMM, we'll use pyhsmm to create a WeakLimitHDPHSMM instance using some reasonable hyperparameters. We'll ask this model to infer the number of states as well, so we'll give it an Nmax parameter:

import pyhsmm
import pyhsmm.basic.distributions as distributions

obs_dim = 2
Nmax = 25

obs_hypparams = {'mu_0':np.zeros(obs_dim),
                'sigma_0':np.eye(obs_dim),
                'kappa_0':0.3,
                'nu_0':obs_dim+5}
dur_hypparams = {'alpha_0':2*30,
                 'beta_0':2}

obs_distns = [distributions.Gaussian(**obs_hypparams) for state in range(Nmax)]
dur_distns = [distributions.PoissonDuration(**dur_hypparams) for state in range(Nmax)]

posteriormodel = pyhsmm.models.WeakLimitHDPHSMM(
        alpha=6.,gamma=6., # better to sample over these; see concentration-resampling.py
        init_state_concentration=6., # pretty inconsequential
        obs_distns=obs_distns,
        dur_distns=dur_distns)

(The first two arguments set the "new-table" proportionality constant for the meta-Chinese Restaurant Process and the other CRPs, respectively, in the HDP prior on transition matrices. For this example, they really don't matter at all, but on real data it's much better to infer these parameters, as in examples/concentration_resampling.py.)

Then, we add the data we want to condition on:

posteriormodel.add_data(data,trunc=60)

The trunc parameter is an optional argument that can speed up inference: it sets a truncation limit on the maximum duration for any state. If you don't pass in the trunc argument, no truncation is used and all possible state duration lengths are considered. (pyhsmm has fancier ways to speed up message passing over durations, but they aren't documented.)

If we had multiple observation sequences to learn from, we could add them to the model just by calling add_data() for each observation sequence.

Now we run a resampling loop. For each iteration of the loop, all the latent variables of the model will be resampled by Gibbs sampling steps, including the transition matrix, the observation means and covariances, the duration parameters, and the hidden state sequence. We'll also copy some samples so that we can plot them.

models = []
for idx in progprint_xrange(150):
    posteriormodel.resample_model()
    if (idx+1) % 10 == 0:
        models.append(copy.deepcopy(posteriormodel))

Now we can plot our saved samples:

fig = plt.figure()
for idx, model in enumerate(models):
    plt.clf()
    model.plot()
    plt.gcf().suptitle('HDP-HSMM sampled after %d iterations' % (10*(idx+1)))
    plt.savefig('iter_%.3d.png' % (10*(idx+1)))

Sampled models

I generated these data from an HSMM that looked like this:

Randomly-generated model and data

So the posterior samples look pretty good!

A convenient shortcut to build a list of sampled models is to write

model_samples = [model.resample_and_copy() for itr in progprint_xrange(150)]

That will build a list of model objects (each of which can be inspected, plotted, pickled, etc, independently) in a way that won't duplicate data that isn't changed (like the observations or hyperparameter arrays) so that memory usage is minimized. It also minimizes file size if you save samples like

import cPickle
with open('sampled_models.pickle','w') as outfile:
    cPickle.dump(model_samples,outfile,protocol=-1)

Extending the Code

To add your own observation or duration distributions, implement the interfaces defined in basic/abstractions.py. To get a flavor of the style, see pybasicbayes.

References

@article{johnson2013hdphsmm,
    title={Bayesian Nonparametric Hidden Semi-Markov Models},
    author={Johnson, Matthew J. and Willsky, Alan S.},
    journal={Journal of Machine Learning Research},
    pages={673--701},
    volume={14},
    month={February},
    year={2013},
}

Authors

Matt Johnson, Alex Wiltschko, Yarden Katz, Chia-ying (Jackie) Lee, Scott Linderman, Kevin Squire, Nick Foti.

Owner
Matthew Johnson
research scientist @ Google Brain
Matthew Johnson
Comments
  • pyhsmm on windows

    pyhsmm on windows

    Hi, After a few days of various combinations of python environments and difficulties with the pyhsmm install, I thought I would check in to see if I there was something I could do that would be more successful. Here's the steps I've taken:

    1. Install Anaconda-2.0.1-Windows-x86_64
    2. Download pyhsmm
    3. Install pyhsmm: python setup.py build_ext --inplace
    4. Errors during compile of first cpp file: hmm_messages_interface.cpp: "::hypot not defined"
    5. From http://bugs.python.org/issue11566: add #include cmath earlier than python.h, i.e., at the top of hmm_messages_interface.cpp
    6. "::hypot" error no longer appears, but the next one remains: "error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive]"

    I don't see a quick fix to that one in the searches that I've done.

    Below is the full install console output. You will notice that I am using the gcc that comes with Anaconda (4.7.0). You do suggest using 4.8 and a quick check of using that instead gave the same errors.

    Any thoughts would be most appreciated, Phil

    running build_ext building 'internals.hmm_messages_interface' extension D:\Anaconda\Scripts\gcc.bat -DMS_WIN64 -mdll -O -Wall -Ideps/Eigen3/ -ID:\Anaconda\lib\site-packages\numpy\core\include -ID:\Anaconda\include -ID:\Anaconda\PC -c internals\hmm_messages_interface.cpp -o build\temp.win-amd64-2.7\Release\internals\hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP -DTEMPS_ON_STACK In file included from D:\Anaconda\lib\site-packages\numpy\core\include/numpy/ndarraytypes.h:1761:0, from D:\Anaconda\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:17, from D:\Anaconda\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4, from internals\hmm_messages_interface.cpp:351: D:\Anaconda\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h:13:79: note: #pragma message: D:\Anaconda\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION In file included from d:\anaconda\mingw\bin../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/include/c++/4.7.0/complex:46:0, from deps/Eigen3/Eigen/Core:56, from internals\hmm_messages.h:4, from internals\hmm_messages_interface.cpp:359: d:\anaconda\mingw\bin../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/include/c++/4.7.0/cmath:1096:11: error: '::hypot' has not been declared internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_68sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:10987:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_70sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:11265:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_72sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:11543:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_74sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:11821:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_76sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12099:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_78sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12377:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_80sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12655:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_82sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12933:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_84sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:13211:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_86sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:13489:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_88sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:13767:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_90sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14045:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_92sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14323:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_94sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14601:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_96sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14879:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_98sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:15157:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_266sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:44761:645: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_268sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:45004:650: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_270sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:45247:645: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_272sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:45490:650: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_286viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:50903:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_288viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51099:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_290viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51295:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_292viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51491:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_294viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51687:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_296viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51883:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_298viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:52079:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_300viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:52275:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] error: command 'D:\Anaconda\Scripts\gcc.bat' failed with exit status 1

  • Question on parallelism

    Question on parallelism

    Hi, A quick question on the parallelism implemented for you code. i am not seen much speed up on some test data so I am wondering if I am understanding how to exploit properly your parallel implementation.

    i am starting the ipcluster with $ ipcluster start --n=8 2013-04-02 14:04:54,751.751 [IPClusterStart] Using existing profile dir: u'/home/inti/.config/ipython/profile_default' 2013-04-02 14:04:54.754 [IPClusterStart] Starting ipcluster with [daemon=False] 2013-04-02 14:04:54.754 [IPClusterStart] Creating pid file: /home/inti/.config/ipython/profile_default/pid/ipcluster.pid 2013-04-02 14:04:54.754 [IPClusterStart] Starting Controller with LocalControllerLauncher 2013-04-02 14:04:55.754 [IPClusterStart] Starting 8 Engines with LocalEngineSetLauncher 2013-04-02 14:05:26.478 [IPClusterStart] Engines appear to have started successfully

    Is the parallelism being applied to each data loaded to the model? i.e., each data loaded with model.add_data_parallel() is being run on a different ipython engine.

    Or is the actual state sampling calculation being spread across the engines?

    thanks in advance,

    BW, Inti

  • import error for pyhsmm examples

    import error for pyhsmm examples

    Hey Matt, thanks for your work here.

    I'm trying to run some of the examples and I'm getting ImportError: No module named hmm_messages_interface.

    Full trace is here:


    python hmm.py
    /Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/transitions.py:16: UserWarning: using slow transition counting
      warn('using slow transition counting')
    
    This demo shows how HDP-HMMs can fail when the underlying data has state
    persistence without some kind of temporal regularization (in the form of a
    sticky bias or duration modeling): without setting the number of states to be
    the correct number a priori, lots of extra states can be intsantiated.
    
    BUT the effect is much more relevant on real data (when the data doesn't exactly
    fit the model). Maybe this demo should use multinomial emissions...
    
    Traceback (most recent call last):
      File "hmm.py", line 64, in <module>
        posteriormodel.resample_model()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/models.py", line 431, in resample_model
        self.resample_states(num_procs=num_procs)
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/models.py", line 456, in resample_states
        s.resample()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 356, in resample
        return self.resample_normalized()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 352, in resample_normalized
        alphan = self.messages_forwards_normalized()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 342, in messages_forwards_normalized
        self._messages_forwards_normalized(self.trans_matrix,self.pi_0,self.aBl)
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 605, in _messages_forwards_normalized
        from hmm_messages_interface import messages_forwards_normalized
    ImportError: No module named hmm_messages_interface
    

    I think this must be related to an issue I had running python setup.py install. While I am able to import pyhsmm, the install command had an error in it:

    building 'pyhsmm.internals.hmm_messages_interface' extension
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -Ideps/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/usr/local/lib/python2.7/site-packages/numpy/core/include -c pyhsmm/internals/hmm_messages_interface.cpp -o build/temp.macosx-10.10-intel-2.7/pyhsmm/internals/hmm_messages_interface.o -std=c++11 -O3 -w -DNDEBUG -DHMM_TEMPS_ON_HEAP
    pyhsmm/internals/hmm_messages_interface.cpp:5484:672: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_20, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:273:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:5875:680: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_20, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:273:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:9632:452: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_17, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:283:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:9988:456: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_17, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:283:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:10818:442: error: cannot initialize
          a parameter of type 'int *' with an rvalue of type '__pyx_t_5numpy_int32_t
          *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_6, __pyx_pybuffernd_stateseq.diminfo[0].strides))));
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:288:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:11014:448: error: cannot initialize
          a parameter of type 'int *' with an rvalue of type '__pyx_t_5numpy_int32_t
          *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_6, __pyx_pybuffernd_stateseq.diminfo[0].strides))));
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:288:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq)
                         ^
    6 errors generated.
    setup.py:35: UserWarning: Failed to build extension modules
      warn('Failed to build extension modules')
    

    For gcc I have

    gcc -v
    Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
    Apple LLVM version 7.0.0 (clang-700.0.72)
    Target: x86_64-apple-darwin14.5.0
    Thread model: posix
    

    Is there something else I need to do with python setup.py install? Thanks!

  • problem compiling pyhsmm on Mac OS X 10.9.4

    problem compiling pyhsmm on Mac OS X 10.9.4

    Hi,

    I'm having trouble compiling pyhsmm on Mac OS X 10.9.4, using the default Apple clang compiler (which if I understood correctly the README says it is supported, although g++ is favored).

    I recursively cloned pyhsmm and then installed it in my local virtualenv, using:

    $ python setup.py build_ext --inplace
    

    which results in:

    $ python setup.py build_ext --inplace
    running build_ext
    building 'internals.hmm_messages_interface' extension
    creating build
    creating build/temp.macosx-10.9-intel-2.7
    creating build/temp.macosx-10.9-intel-2.7/internals
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -Ideps/Eigen3/ -I/Users/yarden/Software/pyenv/lib/python2.7/site-packages/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.9-intel-2.7/internals/hmm_messages_interface.o -Ofast -std=c++11 -DHMM_TEMPS_ON_HEAP -DNDEBUG -w
    internals/hmm_messages_interface.cpp:10976:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:11254:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:11532:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:11810:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12088:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12366:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12644:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12922:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:13200:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:13478:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:13756:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14034:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14312:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14590:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14868:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:15146:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:44750:385: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:44993:389: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:45236:385: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 errors generated.
    error: command 'cc' failed with exit status 1
    

    My version information:

    $ python --version
    Python 2.7.5
    $ python -c 'import numpy; print numpy.__version__'
    1.8.2
    

    Any thoughts on this? Thanks! Yarden

  • "Unexpected keyword argument" and "ndarray is not C contiguous"

    I'm trying to use this for some speaker diarization. I started getting some errors, I checked the examples to make sure it's not just my code. Here's a couple tracebacks:

    Traceback (most recent call last):
      File "concentration-resampling.py", line 45, in <module>
        posteriormodel.resample_model()
      File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 201, in resample_model
        self.resample_parameters(joblib_jobs=obs_jobs)
    TypeError: resample_parameters() got an unexpected keyword argument 'joblib_jobs'
    

    If I delete the parameter on that line, I get another error (included at the bottom because it's long). Since it was related to pybasicbayes I tried reverting to old commits, but none of them seem to work.

    Any ideas? I'm on Linux, as far as I know all my libraries are freshly installed and up to date.

    Traceback (most recent call last): File "concentration-resampling.py", line 45, in posteriormodel.resample_model() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 200, in resample_model self.resample_parameters() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 706, in resample_parameters super(_HSMMGibbsSampling,self).resample_parameters() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 205, in resample_parameters self.resample_trans_distn() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 217, in resample_trans_distn self.trans_distn.resample([s.stateseq for s in self.states_list]) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/internals/transitions.py", line 374, in resample self._resample_beta(ms) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/internals/transitions.py", line 382, in _resample_beta self.beta_obj.resample(ms) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 2059, in resample self.alpha_0_obj.resample(counts) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 3225, in resample return super(GammaCompoundDirichlet,self).resample(data,niter=niter) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 3141, in resample a_n, b_n = self._posterior_hypparams(*self._get_statistics(data)) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 3237, in _get_statistics m = sample_crp_tablecounts(self.concentration,counts,self.weighted_cols) File "pyhsmm/basic/pybasicbayes/util/cstats.pyx", line 50, in pyhsmm.basic.pybasicbayes.util.cstats.sample_crp_tablecounts (basic/pybasicbayes/util/cstats.c:7911) File "pyhsmm/stringsource", line 614, in View.MemoryView.memoryview_cwrapper (basic/pybasicbayes/util/cstats.c:15841) File "pyhsmm/stringsource", line 321, in View.MemoryView.memoryview.cinit (basic/pybasicbayes/util/cstats.c:12387) ValueError: ndarray is not C-contiguous

  • Parallel resampling of observation distributions

    Parallel resampling of observation distributions

    Hi Matt, I have created a resample_model_parallel2 function which resamples the observation distributions in parrallel. This was at the cost of pushing the model into the engines an additional time. However, on my test this had very minor overload. I have added code to parallel.py .. that was easy!!! given the code you had in there already.

    In my initial test with Nmax = 100 the execution time came down to half :)

    i did not see a way to parallelise the sampling of the trans_dist but let me know if you think it is possible I can work on it. resampling of the initial state is super quick anyways so no point in speeding that up.

    Thanks a lot!

    Inti

  • Failed to install pyhsmm on Mac

    Failed to install pyhsmm on Mac

    Below is my problem, I tried all the methods of installing pyhsmm, but still cannot install it on my Mac.

    [email protected]:~/pyhsmm/examples$ pip install pyhsmm Collecting pyhsmm Using cached pyhsmm-0.1.4.tar.gz Requirement already satisfied (use --upgrade to upgrade): Cython>=0.20.1 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): numpy in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): scipy in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): matplotlib in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): nose in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pybasicbayes>=0.1.3 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): six>=1.4 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pytz in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pyparsing>=1.5.6 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): mock in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): future in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pybasicbayes>=0.1.3->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from mock->matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): funcsigs in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from mock->matplotlib->pyhsmm) Building wheels for collected packages: pyhsmm Running setup.py bdist_wheel for pyhsmm Complete output from command /Users/danqing0703/anaconda/bin/python -c "import setuptools;file='/private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" bdist_wheel -d /var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/tmpsYliUapip-wheel-: running bdist_wheel running build running build_py creating build creating build/lib.macosx-10.5-x86_64-2.7 creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm copying pyhsmm/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm copying pyhsmm/models.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm copying pyhsmm/parallel.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/abstractions.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/distributions.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/models.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/hmm_states.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/hsmm_inb_states.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/hsmm_states.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/initial_state.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/transitions.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/plugins copying pyhsmm/plugins/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/plugins creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/general.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/plot.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/profiling.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/stats.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/testing.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/text.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util running build_ext building 'pyhsmm/internals.hmm_messages_interface' extension creating build/temp.macosx-10.5-x86_64-2.7 creating build/temp.macosx-10.5-x86_64-2.7/pyhsmm creating build/temp.macosx-10.5-x86_64-2.7/pyhsmm/internals gcc -fno-strict-aliasing -I/Users/danqing0703/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Ipyhsmm/deps/Eigen3 -I/Users/danqing0703/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/danqing0703/anaconda/include/python2.7 -c pyhsmm/internals/hmm_messages_interface.cpp -o build/temp.macosx-10.5-x86_64-2.7/pyhsmm/internals/hmm_messages_interface.o -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:276: pyhsmm/deps/Eigen3/Eigen/src/Core/MathFunctions.h:357:18: error: no member named 'log1p' in namespace 'std'; did you mean 'log10'? using std::log1p; ~~~~~^~~~~ log10 /usr/include/math.h:391:15: note: 'log10' declared here extern double log10(double); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:324: pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:302:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:303:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:304:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:373:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:374:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:439:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:440:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:329: pyhsmm/deps/Eigen3/Eigen/src/Core/PlainObjectBase.h:470:25: error: no member named 'move' in namespace 'std' : m_storage( std::move(other.m_storage) ) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:222:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:331: pyhsmm/deps/Eigen3/Eigen/src/Core/Array.h:134:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/internals/hmm_messages.h:224:18: note: in instantiation of template class 'Eigen::Matrix<int, -1, -1, 0, -1, -1>' requested here MatrixXi args(M,T); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::base; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:144:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::coeffRef; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:378:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::m_storage; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:73:20: note: in instantiation of template class 'Eigen::Matrix<float, -1, -1, 1, -1, -1>' requested here typedef typename PlainObjectType::Index Index; ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:32:31: note: in instantiation of template class 'Eigen::internal::traits<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here enum { has_direct_access = (traits::Flags & DirectAccessBit) ? 1 : 0, ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:109:32: note: in instantiation of template class 'Eigen::internal::accessors_level<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here int Level = internal::accessors_level::has_write_access ? WriteAccessors : ReadOnlyAccessors ^ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:105:12: note: in instantiation of default argument for 'MapBase<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' required here : public MapBase<Map<PlainObjectType, MapOptions, StrideType> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/internals/hmm_messages.h:31:24: note: in instantiation of template class 'Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> >' requested here NPMatrix eA(A,M,M); ^ pyhsmm/internals/hmm_messages.h:260:12: note: in instantiation of function template specialization 'hmm::messages_backwards_log' requested here { hmm::messages_backwards_log(M,T,A,aBl,betal); } ^ pyhsmm/internals/hmm_messages_interface.cpp:2961:15: note: in instantiation of member function 'hmmc<float, int>::messages_backwards_log' requested here __pyx_v_ref.messages_backwards_log((__pyx_v_A.shape[0]), (__pyx_v_aBl.shape[0]), (&(((float *) ( / dim=1 / ((char *) (((float *) ( / dim=0 / (__pyx_v_A.data + __pyx_t_1 * __pyx_v_A.strides[0]) )) + __pyx_t_2)) )))), (&(((float ) ( / dim=1 / ((char *) (((float *) ( / dim=0 _/ (__pyx_v_aBl.data + __pyx_t_3 * __pyx_v_aBl.strides[0]) )) + __pyx_t_4)) )))), (&(___Pyx_BufPtrCContig2d(float *, __pyx_pybuffernd_betal.rcbuffer->pybuffer.buf, __pyx_t_5, __pyx_pybuffernd_betal.diminfo[0].strides, __pyx_t_6, __pyx_pybuffernd_betal.diminfo[1].strides)))); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' using Base::base; ^~~~~~ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'gcc' failed with exit status 1


    Failed building wheel for pyhsmm Failed to build pyhsmm Installing collected packages: pyhsmm Running setup.py install for pyhsmm Complete output from command /Users/danqing0703/anaconda/bin/python -c "import setuptools, tokenize;file='/private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-5FdwS0-record/install-record.txt --single-version-externally-managed --compile: running install running build running build_py running build_ext building 'pyhsmm/internals.hmm_messages_interface' extension gcc -fno-strict-aliasing -I/Users/danqing0703/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Ipyhsmm/deps/Eigen3 -I/Users/danqing0703/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/danqing0703/anaconda/include/python2.7 -c pyhsmm/internals/hmm_messages_interface.cpp -o build/temp.macosx-10.5-x86_64-2.7/pyhsmm/internals/hmm_messages_interface.o -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:276: pyhsmm/deps/Eigen3/Eigen/src/Core/MathFunctions.h:357:18: error: no member named 'log1p' in namespace 'std'; did you mean 'log10'? using std::log1p; ~~~~~^~~~~ log10 /usr/include/math.h:391:15: note: 'log10' declared here extern double log10(double); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:324: pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:302:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:303:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:304:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:373:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:374:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:439:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:440:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:329: pyhsmm/deps/Eigen3/Eigen/src/Core/PlainObjectBase.h:470:25: error: no member named 'move' in namespace 'std' : m_storage( std::move(other.m_storage) ) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:222:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:331: pyhsmm/deps/Eigen3/Eigen/src/Core/Array.h:134:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/internals/hmm_messages.h:224:18: note: in instantiation of template class 'Eigen::Matrix<int, -1, -1, 0, -1, -1>' requested here MatrixXi args(M,T); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::base; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:144:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::coeffRef; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:378:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::m_storage; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:73:20: note: in instantiation of template class 'Eigen::Matrix<float, -1, -1, 1, -1, -1>' requested here typedef typename PlainObjectType::Index Index; ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:32:31: note: in instantiation of template class 'Eigen::internal::traits<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here enum { has_direct_access = (traits::Flags & DirectAccessBit) ? 1 : 0, ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:109:32: note: in instantiation of template class 'Eigen::internal::accessors_level<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here int Level = internal::accessors_level::has_write_access ? WriteAccessors : ReadOnlyAccessors ^ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:105:12: note: in instantiation of default argument for 'MapBase<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' required here : public MapBase<Map<PlainObjectType, MapOptions, StrideType> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/internals/hmm_messages.h:31:24: note: in instantiation of template class 'Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> >' requested here NPMatrix eA(A,M,M); ^ pyhsmm/internals/hmm_messages.h:260:12: note: in instantiation of function template specialization 'hmm::messages_backwards_log' requested here { hmm::messages_backwards_log(M,T,A,aBl,betal); } ^ pyhsmm/internals/hmm_messages_interface.cpp:2961:15: note: in instantiation of member function 'hmmc<float, int>::messages_backwards_log' requested here __pyx_v_ref.messages_backwards_log((__pyx_v_A.shape[0]), (__pyx_v_aBl.shape[0]), (&(((float *) ( / dim=1 / ((char *) (((float *) ( / dim=0 / (__pyx_v_A.data + __pyx_t_1 * __pyx_v_A.strides[0]) )) + __pyx_t_2)) )))), (&(((float ) ( / dim=1 / ((char *) (((float *) ( / dim=0 _/ (__pyx_v_aBl.data + __pyx_t_3 * __pyx_v_aBl.strides[0]) )) + __pyx_t_4)) )))), (&(___Pyx_BufPtrCContig2d(float *, __pyx_pybuffernd_betal.rcbuffer->pybuffer.buf, __pyx_t_5, __pyx_pybuffernd_betal.diminfo[0].strides, __pyx_t_6, __pyx_pybuffernd_betal.diminfo[1].strides)))); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' using Base::base; ^~~~~~ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'gcc' failed with exit status 1

    ----------------------------------------
    

    Command "/Users/danqing0703/anaconda/bin/python -c "import setuptools, tokenize;file='/private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-5FdwS0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm

  • Fixup for pypi distribution of pyhsmm

    Fixup for pypi distribution of pyhsmm

    I missed one wrinkle with pypi distribution. The previous version didn't include the *.pyx files in the distribution, and so the package from pypi would install correctly but would not compile the Cython files.

    For now, I included the Cython files and associated headers through MANIFEST.in. This will make it so when you install the package from pypi, the installation procedure automatically compiles the Cython files. A more ideal solution would be to write a conditional in the setup.py that checks if Cython is present, and if so, compiles the Cython files. For developers who want to make a distribution with python setup.py sdist, only the Cython-generated *.c files would go in the distribution, and then users could install with just Python and a C compiler, without needing Cython. (This more ideal setup is described here: http://stackoverflow.com/questions/4505747/how-should-i-structure-a-python-package-that-contains-cython-code). Since your users are mostly expert developers who probably have Cython, I figured it's not pressing.

    One caveat to watch out for: if you compile your pyx files to make *.c files, and have something that includes all *.c files in MANIFEST.in, you could accidentally package a Cython generated, e.g. foo.c along with a foo.pyx file. That could might create a situation where only the foo.c gets used even if it's out of date with the foo.pyx file. So to be on safe side, best to wipe out all Cython generated files before making a source distribution, i.e. before running python setup.py sdist.

    I fixed this in pybasicbayes too, see other pull request.

    It'd be great if you could make a fresh source distribution and upload it to pypi. You could wipe out the previous one through the pypi admin settings, or upgrade the VERSION variable in setup.py and make a new version release. I'm sure that if you wipe out the previous one and replace it without changing the version, you won't upset anyone, it has only been out for a few hours :)

  • pyhsmm compilation issue on windows

    pyhsmm compilation issue on windows

    Hi Guys,

    I am trying to compile pyhsmm with windows I am getting error,

    Please guide me How to fix this.

    My Environment ::

    Gnu Python-2.7 and [ Microsoft Visual Studio 11.0 ] windows 7 ( 64-bit ) Operating system.

    C:\Users\localadmin\pyhsmm> C:\Python27\python.exe .\setup.py build_ext --inplace nning build_ext ilding 'internals.hmm_messages_interface' extension eating build eating build\temp.win32-2.7 eating build\temp.win32-2.7\Release eating build\temp.win32-2.7\Release\internals \Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ideps/Eigen3/ -I \Python27\lib\site-packages\numpy\core\include -IC:\Python27\include -IC:\Python27\PC /Tpinternals\hmm_messages_interf e.cpp /Fobuild\temp.win32-2.7\Release\internals\hmm_messages_interface.obj -Ofast -std=c++11 -DHMM_TEMPS_ON_HEAP -DNDE G -w : Command line warning D9025 : overriding '/Os' with '/Ot' : Command line warning D9025 : overriding '/W3' with '/w' : Command line warning D9002 : ignoring unknown option '-Of' : Command line warning D9002 : ignoring unknown option '-Oa' : Command line warning D9002 : ignoring unknown option '-std=c++11' m_messages_interface.cpp \python27\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumP API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION \Program Files (x86)\Microsoft Visual Studio 11.0\VC\INCLUDE\xlocale(336) : warning C4530: C++ exception handler used, ut unwind semantics are not enabled. Specify /EHsc \users\localadmin\pyhsmm\internals\nptypes.h(24) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(24) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(30) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(30) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(36) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(36) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(61) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(61) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(67) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(67) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(73) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(73) : error C2059: syntax error : 'using' ternals\hmm_messages_interface.cpp(5120) : error C2664: 'hmmc::sample_forwards_log' : cannot convert parame r 7 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=float ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(5419) : error C2664: 'hmmc::sample_forwards_log' : cannot convert parame r 7 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=double ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(8753) : error C2664: 'hmmc::sample_backwards_normalized' : cannot conver parameter 5 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=float ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(9017) : error C2664: 'hmmc::sample_backwards_normalized' : cannot conver parameter 5 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=double ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(12382) : error C2664: 'hmmc::viterbi' : cannot convert parameter 6 from _pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=float ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(12578) : error C2664: 'hmmc::viterbi' : cannot convert parameter 6 from _pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=double ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ror: command '"C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\cl.exe"' failed with exit status 2 C:\Users\localadmin\pyhsmm>

  • Can't compile on OS X 10.9.3

    Can't compile on OS X 10.9.3

    Hi,

    I'm running into a similar issue as https://github.com/mattjj/pyhsmm/issues/21. I'm on OS X 10.9.3, and I'm using clang (Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)).

    I tried building in two ways specified by the README but both failed for me.

    First I tried:

    $ python setup.py build_ext --inplace running build_ext gcc-4.2 not found, using clang instead building 'internals.hmm_messages_interface' extension clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP In file included from internals/hmm_messages_interface.cpp:360: In file included from internals/hmm_messages.h:4: In file included from deps/Eigen3/Eigen/Core:287: deps/Eigen3/Eigen/src/Core/MathFunctions.h:357:18: error: no member named 'log1p' in namespace 'std'; did you mean 'log10'? using std::log1p; ~~~~~^~~~~ log10 ... bunch of other errors follow ...

    I also tried the suggested alternative for clang but got: $ python setup.py build_ext --inplace --with-old-clang running build_ext gcc-4.2 not found, using clang instead building 'internals.hmm_messages_interface' extension clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP -stdlib=libc++ clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) error: command 'clang' failed with exit status 1

    Any ideas?

    Thanks

  • error with discrete observations in pyhsmm

    error with discrete observations in pyhsmm

    I'm trying to run pyhsmm on discrete (categorical) observations. Sampling runs, but using the sampled models gives errors. Code:

    Nmax = 25
    # observations
    obs_hypparams = {"K": 2,
                     "alpha_0": 1.}
    # gamma prior on duration 
    dur_hypparams = {'alpha_0': 1.,
                     'beta_0': 1.}
    obs_distns = [distributions.Categorical(**obs_hypparams) for state in range(Nmax)]
    dur_distns = [distributions.PoissonDuration(**dur_hypparams) for state in range(Nmax)]
    
    posteriormodel = pyhsmm.models.WeakLimitHDPHSMM(
            alpha=6.,gamma=6.,
            init_state_concentration=6.,
            obs_distns=obs_distns,
            dur_distns=dur_distns)
    
    # data
    data = np.array([0, 1] * 10 + [0, 0] * 10)
    posteriormodel.add_data(data)
    
    # inference
    models = []
    for idx in progprint_xrange(150):
        posteriormodel.resample_model()
        if (idx+1) % 10 == 0:
            models.append(copy.deepcopy(posteriormodel))
    
    # try to plot model (this fails)
    fig = plt.figure()
    for idx, model in enumerate(models):
        plt.clf()
        model.plot()
        plt.gcf().suptitle('HDP-HSMM sampled after %d iterations' % (10*(idx+1)))
    
    # try to predict (this fails)
    m = models[0]
    m.predict(np.array([]), 5)
    

    Prediction (m.predict) fails with the error:

        111     def predict(self,seed_data,timesteps,**kwargs):
    --> 112         full_data = np.vstack((seed_data,np.nan*np.ones((timesteps,seed_data.shape[1]))))
        113         self.add_data(full_data,**kwargs)
        114         s = self.states_list.pop()
    
    IndexError: tuple index out of range
    

    and plotting fails with:

    models.pyc in _plot_2d_data_scatter(self, ax, state_colors, plot_slice, update)
        308                 s._data_scatter.set_color(colorseq)
        309             else:
    --> 310                 s._data_scatter = ax.scatter(data[:,0],data[:,1],c=colorseq,s=5)
        311             artists.append(s._data_scatter)
        312 
    
    IndexError: too many indices for array
    

    I don't care about using plotting; just using it to test if model worked. I mainly want to call predict on the sampled models. Any thoughts on how to run pyhsmm with discrete observations? Thanks!

  • Installing on linux (HPCC)

    Installing on linux (HPCC)

    I had some mild trouble installing pyhsmm on a linux-based computing cluster...documenting various errors and fixes in one place for anyone else who encounters them. Most have been solved in the issues before, but there are a few key points.

    1. You need to have the packages cython, requests, and futures installed before you run pyhsmm installation. (This is because the setup.py script imports them -- the script isn't smart enough to look in its own list of requirements before importing the modules it uses :).)
    2. Don't use python setup.py install, use pip install . This is critical. (See discussions e.g. here).
    3. You have to patch pybasicbayes manually (at least until the latest fix gets pushed to pypi)

    Full list of commands I ran: conda create -n pyhsmm_test_2 python=3.7 conda activate pyhsmm_test_2 pip install cython pip install requests pip install future cd [where you want the pyhsmm repo to live] git clone https://github.com/mattjj/pyhsmm.git cd pyhsmm pip install . pip install --upgrade git+https://github.com/mattjj/pybasicbayes.git

    And then double-check with: python /path/to/pyhsmm/examples/hmm.py

  • Couldn't install pyhsmm on win 10: ERROR: Command errored out with exit status 1

    Couldn't install pyhsmm on win 10: ERROR: Command errored out with exit status 1

    Hi every one. I learned about this amazing hsmm library for python from a scientific paper. I now really need to use it. I tried to install it on my windows 10 laptop and pc. and none of them worked out. I searched every error I got and tried to solve them using previous issues.

    GCC

    I installed gcc using these recomendations. Then, this is my installed gcc version:

    gcc (MinGW.org GCC-6.3.0-1) 6.3.0
    Copyright (C) 2016 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    

    Microsoft Visual Studio 14.0

    I got errors regarding Ms VS14.0, so I downloaded ms build tools from Microsoft VisualStudio Downloads website and installed the MSVC v14.0 - VS 2015 C++ build tools (v14.00). I tried both enterprise and community version of vs installer.

    pyhsmm Installation

    then I tried installing the package using pip:

    pip install pyhsmm
    

    now here is what I get:

    (base) C:\Windows\system32>pip install pyhsmm
    Collecting pyhsmm
      Using cached pyhsmm-0.1.7.tar.gz (513 kB)
        ERROR: Command errored out with exit status 1:
         command: 'C:\ProgramData\Anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = 
    '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_322beb62dbe646499d9798cd3d759edd\\setup.py'"'"';
     __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
    ox7amkxz\\pyhsmm_322beb62dbe646499d9798cd3d759edd\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
    (__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 
    'C:\Users\USER\AppData\Local\Temp\pip-pip-egg-info-6ehmj4dx'
             cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_322beb62dbe646499d9798cd3d759edd\
        Complete output (26 lines):
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_322beb62dbe646499d9798cd3d759edd\setup.py", line
     76, in <module>
            urlretrieve(eigenurl, eigentarpath)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 247, in urlretrieve
            with contextlib.closing(urlopen(url, data)) as fp:
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 222, in urlopen
            return opener.open(url, data, timeout)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 525, in open
            response = self._open(req, data)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 542, in _open
            result = self._call_chain(self.handle_open, protocol, protocol +
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 502, in _call_chain
            result = func(*args)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1383, in http_open
            return self.do_open(http.client.HTTPConnection, req)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1358, in do_open
            r = h.getresponse()
          File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1347, in getresponse
            response.begin()
          File "C:\ProgramData\Anaconda3\lib\http\client.py", line 307, in begin
            version, status, reason = self._read_status()
          File "C:\ProgramData\Anaconda3\lib\http\client.py", line 276, in _read_status
            raise RemoteDisconnected("Remote end closed connection without"
        http.client.RemoteDisconnected: Remote end closed connection without response
        Downloading Eigen...
        ----------------------------------------
    WARNING: Discarding https://files.pythonhosted.org/packages/2c/77/6d232566005a1ab9662f903e45122ce4e3679e881a8d0cb0d670c05a49bf/pyhsmm-0.1.7.tar.gz#sha256=3677225656bc39f7d9888361cfeba2c938437ba1df6c8d801985df040fb55f7a 
    (from https://pypi.org/simple/pyhsmm/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full 
    command output.
      Using cached pyhsmm-0.1.6.tar.gz (491 kB)
        ERROR: Command errored out with exit status 1:
         command: 'C:\ProgramData\Anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = 
    '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_8899810deae1422dbe94c49cf35655af\\setup.py'"'"'; 
    __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
    ox7amkxz\\pyhsmm_8899810deae1422dbe94c49cf35655af\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
    (__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 
    'C:\Users\USER\AppData\Local\Temp\pip-pip-egg-info-88pfdwxz'
             cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_8899810deae1422dbe94c49cf35655af\
        Complete output (5 lines):
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_8899810deae1422dbe94c49cf35655af\setup.py", line 11, in <module>
            from urllib import urlretrieve
        ImportError: cannot import name 'urlretrieve' from 'urllib' (C:\ProgramData\Anaconda3\lib\urllib\__init__.py)
        ----------------------------------------
    WARNING: Discarding https://files.pythonhosted.org/packages/9d/91/9a6ea2ac2eb82953b253f3b16967a283ebda324923b4989c34df05de02f8/pyhsmm-0.1.6.tar.gz#sha256=1d000d709f9ba63fbb23e0d509f51dc41b5cc079253d6f4fcb7450a841764c3e (from https://pypi.org/simple/pyhsmm/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
      Using cached pyhsmm-0.1.4.tar.gz (2.0 MB)
    Requirement already satisfied: Cython>=0.20.1 in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (0.29.23)
    Requirement already satisfied: numpy in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.20.1)
    Requirement already satisfied: scipy in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.6.2)
    Requirement already satisfied: matplotlib in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (3.3.4)
    Requirement already satisfied: nose in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.3.7)
    Requirement already satisfied: pybasicbayes>=0.1.3 in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (0.2.2)
    Requirement already satisfied: future in c:\programdata\anaconda3\lib\site-packages (from pybasicbayes>=0.1.3->pyhsmm) (0.18.2)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (2.4.7)
    Requirement already satisfied: python-dateutil>=2.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (2.8.1)
    Requirement already satisfied: kiwisolver>=1.0.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (1.3.1)
    Requirement already satisfied: pillow>=6.2.0 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (8.2.0)
    Requirement already satisfied: cycler>=0.10 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (0.10.0)
    Requirement already satisfied: six in c:\programdata\anaconda3\lib\site-packages (from cycler>=0.10->matplotlib->pyhsmm) (1.15.0)
    Building wheels for collected packages: pyhsmm
      Building wheel for pyhsmm (setup.py) ... error
      ERROR: Command errored out with exit status 1:
       command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-bkx1ousu'
           cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\
      Complete output (44 lines):
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-3.8
      creating build\lib.win-amd64-3.8\pyhsmm
      copying pyhsmm\models.py -> build\lib.win-amd64-3.8\pyhsmm
      copying pyhsmm\parallel.py -> build\lib.win-amd64-3.8\pyhsmm
      copying pyhsmm\__init__.py -> build\lib.win-amd64-3.8\pyhsmm
      creating build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\abstractions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\distributions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\models.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      creating build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\hmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\hsmm_inb_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\hsmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\initial_state.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\transitions.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      creating build\lib.win-amd64-3.8\pyhsmm\plugins
      copying pyhsmm\plugins\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\plugins
      creating build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\general.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\plot.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\profiling.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\stats.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\testing.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\text.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\util
      running build_ext
      building 'pyhsmm\internals.hmm_messages_interface' extension
      creating build\temp.win-amd64-3.8
      creating build\temp.win-amd64-3.8\Release
      creating build\temp.win-amd64-3.8\Release\pyhsmm
      creating build\temp.win-amd64-3.8\Release\pyhsmm\internals
      C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ipyhsmm\deps\Eigen3 -IC:\ProgramData\Anaconda3\lib\site-packages\numpy\core\include -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /EHsc /Tppyhsmm\internals\hmm_messages_interface.cpp /Fobuild\temp.win-amd64-3.8\Release\pyhsmm\internals\hmm_messages_interface.obj -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP
      cl : Command line warning D9025 : overriding '/W3' with '/w'
      cl : Command line warning D9002 : ignoring unknown option '-O3'
      cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
      hmm_messages_interface.cpp
      C:\ProgramData\Anaconda3\include\pyconfig.h(205): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
      ----------------------------------------
      ERROR: Failed building wheel for pyhsmm
      Running setup.py clean for pyhsmm
    Failed to build pyhsmm
    Installing collected packages: pyhsmm
        Running setup.py install for pyhsmm ... error
        ERROR: Command errored out with exit status 1:
         command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = 
    '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; 
    __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
    ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
    (__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 
    'C:\Users\USER\AppData\Local\Temp\pip-record-90tew8g2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\pyhsmm'
             cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\
        Complete output (44 lines):
        running install
        running build
        running build_py
        creating build
        creating build\lib.win-amd64-3.8
        creating build\lib.win-amd64-3.8\pyhsmm
        copying pyhsmm\models.py -> build\lib.win-amd64-3.8\pyhsmm
        copying pyhsmm\parallel.py -> build\lib.win-amd64-3.8\pyhsmm
        copying pyhsmm\__init__.py -> build\lib.win-amd64-3.8\pyhsmm
        creating build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\abstractions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\distributions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\models.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        creating build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\hmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\hsmm_inb_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\hsmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\initial_state.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\transitions.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        creating build\lib.win-amd64-3.8\pyhsmm\plugins
        copying pyhsmm\plugins\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\plugins
        creating build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\general.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\plot.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\profiling.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\stats.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\testing.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\text.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\util
        running build_ext
        building 'pyhsmm\internals.hmm_messages_interface' extension
        creating build\temp.win-amd64-3.8
        creating build\temp.win-amd64-3.8\Release
        creating build\temp.win-amd64-3.8\Release\pyhsmm
        creating build\temp.win-amd64-3.8\Release\pyhsmm\internals
        C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ipyhsmm\deps\Eigen3 -IC:\ProgramData\Anaconda3\lib\site-packages\numpy\core\include -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /EHsc /Tppyhsmm\internals\hmm_messages_interface.cpp /Fobuild\temp.win-amd64-3.8\Release\pyhsmm\internals\hmm_messages_interface.obj -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP
        cl : Command line warning D9025 : overriding '/W3' with '/w'
        cl : Command line warning D9002 : ignoring unknown option '-O3'
        cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
        hmm_messages_interface.cpp
        C:\ProgramData\Anaconda3\include\pyconfig.h(205): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
        error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
        ----------------------------------------
    ERROR: Command errored out with exit status 1: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-90tew8g2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\pyhsmm' Check the logs for full command output.
    

    any help is much appreciated.

  • HTTP 404 error when installing in WSL Debian

    HTTP 404 error when installing in WSL Debian

    OS: Windows Subsystem for Linux, Debian Miniconda3 command: pip install pyhsmm Here is the error message: Collecting pyhsmm Using cached pyhsmm-0.1.7.tar.gz (513 kB) ERROR: Command errored out with exit status 1: command: /home/whr/miniconda3/envs/somoseq/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/setup.py'"'"'; file='"'"'/tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vxgwk3u3 cwd: /tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/ Complete output (30 lines): Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/setup.py", line 76, in urlretrieve(eigenurl, eigentarpath) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 248, in urlretrieve with contextlib.closing(urlopen(url, data)) as fp: File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 223, in urlopen return opener.open(url, data, timeout) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 564, in error result = self._call_chain(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 756, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 570, in error return self._call_chain(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found Downloading Eigen... ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

    It turns out that the link used to downloading "Eigen" package is not valid anymore.

  • Anyway to calculate the bayes factor of the model conveniently?

    Anyway to calculate the bayes factor of the model conveniently?

    I am try to tuning the emission function for my data and have some candidate, is there a way to calculate the bayes factor of the pyhsmm outputs? Or in other words calculate the marginal likelihood for a fitmodel?

  • clang: error: unsupported option '-fopenmp'

    clang: error: unsupported option '-fopenmp'

    Hi.

    I wonder what would be the reason for this error.

    I was using pyhsmm/rslds package with an autoregressive extension on my laptop on Mojave without any problem. But now using a new iMac with Catalina (I presume there can be many changes but most prominent change).

    I tried installing and reinstalling Xcode, brew installed gcc, llvm, limomp. However, nothing worked.

    I tried the no-complied version. It seemed installed but giving an error when importing it (at init.py importing models).

    Any solution would be appreciated. Thank you.

    Best, Michael

Related tags
Auto HMM: Automatic Discrete and Continous HMM including Model selection

Auto HMM: Automatic Discrete and Continous HMM including Model selection

Sep 19, 2022
TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction.

TalkNet 2 [WIP] TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Predictio

Aug 28, 2022
Rate-limit-semaphore - Semaphore implementation with rate limit restriction for async-style (any core)
Rate-limit-semaphore - Semaphore implementation with rate limit restriction for async-style (any core)

Rate Limit Semaphore Rate limit semaphore for async-style (any core) There are t

Jun 21, 2022
Exploit Camera Raw Data for Video Super-Resolution via Hidden Markov Model Inference
 Exploit Camera Raw Data for Video Super-Resolution via Hidden Markov Model Inference

RawVSR This repo contains the official codes for our paper: Exploit Camera Raw Data for Video Super-Resolution via Hidden Markov Model Inference Xiaoh

Apr 2, 2022
Information-Theoretic Multi-Objective Bayesian Optimization with Continuous Approximations

Information-Theoretic Multi-Objective Bayesian Optimization with Continuous Approximations Requirements The code is implemented in Python and requires

Nov 3, 2021
Self-Adaptable Point Processes with Nonparametric Time Decays

NPPDecay This is our implementation for the paper Self-Adaptable Point Processes with Nonparametric Time Decays, by Zhimeng Pan, Zheng Wang, Jeff M. P

Jun 16, 2022
Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayesian-Torch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.

Sep 21, 2022
Quick program made to generate alpha and delta tables for Hidden Markov Models

HMM_Calc Functions for generating Alpha and Delta tables from a Hidden Markov Model. Parameters: a: Matrix of transition probabilities. a[i][j] = a_{i

Dec 4, 2021
HiddenMarkovModel implements hidden Markov models with Gaussian mixtures as distributions on top of TensorFlow

Class HiddenMarkovModel HiddenMarkovModel implements hidden Markov models with Gaussian mixtures as distributions on top of TensorFlow 2.0 Installatio

Nov 3, 2021
Ejemplo Algoritmo Viterbi - Example of a Viterbi algorithm applied to a hidden Markov model on DNA sequence
Ejemplo Algoritmo Viterbi - Example of a Viterbi algorithm applied to a hidden Markov model on DNA sequence

Ejemplo Algoritmo Viterbi Ejemplo de un algoritmo Viterbi aplicado a modelo ocul

Jan 10, 2022
A Nim frontend for pytorch, aiming to be mostly auto-generated and internally using ATen.
A Nim frontend for pytorch, aiming to be mostly auto-generated and internally using ATen.

Master Release Pytorch - Py + Nim A Nim frontend for pytorch, aiming to be mostly auto-generated and internally using ATen. Because Nim compiles to C+

Sep 4, 2022
Code for all the Advent of Code'21 challenges mostly written in python

Advent of Code 21 Code for all the Advent of Code'21 challenges mostly written in python. They are not necessarily the best or fastest solutions but j

May 26, 2022
Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

SW-CV-ModelZoo Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset Framework: TF/Keras 2.7 Training SQLite D

Sep 6, 2022
Pytorch Implementation of Google's Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling
Pytorch Implementation of Google's Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling

Parallel Tacotron2 Pytorch Implementation of Google's Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling

Sep 19, 2022
PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.

PyTorch-LIT PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices. With

Sep 13, 2022
SSD: Single Shot MultiBox Detector pytorch implementation focusing on simplicity
SSD: Single Shot MultiBox Detector pytorch implementation focusing on simplicity

SSD: Single Shot MultiBox Detector Introduction Here is my pytorch implementation of 2 models: SSD-Resnet50 and SSDLite-MobilenetV2.

Sep 16, 2022
A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.
A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.

AnimeGAN A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing. Randomly Generated Images The images are

Sep 10, 2022
A Kernel fuzzer focusing on race bugs

Razzer: Finding kernel race bugs through fuzzing Environment setup $ source scripts/envsetup.sh scripts/envsetup.sh sets up necessary environment var

Sep 11, 2022