Software

From SpeechWiki

(Difference between revisions)

Jump to: navigation, search

Latest revision as of 17:19, 4 February 2015

Learning

HTKtrain (Sarah Borys and Mark Hasegawa-Johnson, 2008): source, tgz; Scripts for training HMMs using HTK

Pronounce (Arthur Kantor, 2007): Description, Demo, source, tgz; An orthographic string to phonetic string mapping tool.; This tool computes American English phonetic transcriptions from plaintext. Its HMM either generates a most likely phonetic transcription, or forces alignment if a phonetic transcription is provided. So, it gives a reasonable pronunciation for both out-of-dictionary words and partially pronounced words.

HTK-based Explicit-duration HMM (Ken Chen, 2003): Description, source, tgz

Signal Processing

Singing-Voice Separation From Monaural Recordings Using Robust Principal Component Analysis (Po-Sen Huang, 2012): Projects Description and Demo MATLAB code

Nested STFTs (Dave Cohen, Camille Goudeseune, Mark Hasegawa-Johnson, 2009): Efficient Simultaneous Multi-Scale Computation of FFTs; Description, stft.c

Improved Mistral (Qingsong Liu, 2009)

State of the Art Text-Independent Speaker Verification System,especially for NIST SRE

Based on Mistral Open Source package

Improved and New Features:

full factor analysis(eigenchannel and eigenvoice), instead of simple factor analysis(eigenchannel)
multi-threads for Windows as well as Linux
read HTK format feature/model
effective Algorithm for fast implementation of FA.
code optimization(for FA)
fixed some bugs

Source: /ws/ifp-32-2/hasegawa/pineking/programs/Improved_Mistral

PVTK (Sarah Borys and Mark Hasegawa-Johnson, 2005-8): source, tgz; Extract HTK features as training vectors for libSVM, apply trained SVMs directly to feature files

VAD (Bowon Lee, 2007): Description, source, tgz; Voice activity detector with improved noise model

Computation

Matlab GMM (Arthur Kantor, 2010): source, tgz; A somewhat optimized Matlab toolbox for calculating the likelihood of many observations against many gaussian mixtures, each with many diagonal-covariance componenents.

GMTK Parallel (Arthur Kantor, 2008): Description; Run GMTK commands in parallel on a compute cluster. Email Arthur for code.

HTK Parallel (Bowon Lee, 2006): description, source, tgz; These Perl scripts split an HTK command for parallel excution on an SGE cluster.

Data

Timeliner (Camille Goudeseune, 2012): Description, Linux source tgz; Browser for long audio files, with generated spectrograms and other derived features.

Matlab pfile I/O toolbox (Arthur Kantor, 2010): source, tgz; A Matlab toolbox for reading and writing ICSI pfile data format used by GMTK and QuickNet.; The toolbox is designed to work with large pfiles (hundreds of GB).; It is based on the pfread.m and pfinfo.m scripts by Dan Ellis.

Python library for reading/writing GMTK parameter files (Arthur Kantor, 2010): source, tgz; The library can read/write complete TrainableParameters files, as well as decision trees and most other objects allowed in GMTK parameter files.; This library is based on the code from EHVS parser project, and so is available under the GPL3 license.

Improved MVA (Arthur Kantor, 2008)

Linux binary

Perform mean and variance normalization and ARMA filtering

It's essentially this version but with

better error reporting (e.g. failing to open file tells you so instead of core dumping)
more accurate mean and variance estimation (doubles instead of floats in strategic places)
faster computation in the case of MV (ARMA order 0)

svn location is svn://mickey.ifp.uiuc.edu/corporaNormalizationScripts/fisher/MVA.cc

DTMFseg (Bowon Lee, 2006): source, tgz; Segment audio files at DTMF tones

Transcription tools (Mark Hasegawa-Johnson, 2005): source, tgz; Convert transcription formats

Speechfileformats (Mark Hasegawa-Johnson, 2004): source, tgz; Read and write HTK files in matlab

CTMRedit (Jul Cha and Mark Hasegawa-Johnson, 1999): Description, source, tgz; Manually and automatically segment CT and MR image stacks

LaTeX tools

LaTeX scripts to import figures from dia and pdf. (Arthur Kantor, 2010)

Miscellaneous

Other scripts written in perl, python, bash, and ruby can be found in SVN archive.

There is also auto-generated documentation for them.

@@ Line 1: / Line 1: @@
-= Installing / Arranging Software =
+===Statistical Speech Technology Group Software===
-If you download linux software from the internet, and find it useful, please put it where others may also use it!  Here's how.
+Our policy: everything we write is free on the web. This wiki is intended to be definitive, because anybody in the group can edit it to add their own software. A spider-indexable backup is at http://www.isle.uiuc.edu/software .
-(1) Type `umask 022` or `umask 000`.  If you use 022, you are volunteering to manage the package; if you use 000, you are inviting others to help manage it.
+You can access each project by browsing an SVN snapshot online, or by downloading a tgz file (gzipped .tar file) by using one of the links below.
-(2) Download the tarfile to /workspace/ifp-32-1/hasegawa/programs; untar it to create $PACKAGE_DIR; remove the tar file (important!); configure; make all.
+You can also check it out of our [http://subversion.apache.org subversion] server using login name "anon" with no password (hit "enter" when a password is requested).
+* On Windows, use [http://tortoisesvn.net/ TortoiseSVN].
+* On Linux, use <code>svn</code>.  For example, for a project listed at {{SoftwarePath}}/speechfileformats use the command
+ svn co svn://mickey.ifp.illinois.edu/speechfileformats
-(3) Decide where you want the binaries.  Reasonable places for programs are /workspace/ifp-32-1/hasegawa/programs/...
+==Learning==
+;HTKtrain (Sarah Borys and Mark Hasegawa-Johnson, 2008)
+:[{{SoftwarePath}}/HTKtrain source], [{{SoftwarePath}}/HTKtrain.tgz tgz]
+:Scripts for training HMMs using HTK
-** scripts = executes on any machine (e.g., perl, bash scripts)
+; Pronounce (Arthur Kantor, 2007)
+: [[Phonetic Transcription Tool | Description]], [http://mickey.ifp.uiuc.edu/speech/webpronounce/webpronounce.cgi Demo], [{{SoftwarePath}}/pronounce source], [{{SoftwarePath}}/pronounce.tgz tgz]
+: An orthographic string to phonetic string mapping tool.
+:This tool computes American English phonetic transcriptions from plaintext.  Its HMM either generates a most likely phonetic transcription, or forces alignment if a phonetic transcription is provided.  So, it gives a reasonable pronunciation for both out-of-dictionary words and partially pronounced words.
-** bin.`uname` (i.e., bin.Linux) = executes on both ifp-32 and mickey.  PLEASE CHECK: ssh mickey; execute code; see if it gives you "cannot execute binary file".
+;HTK-based Explicit-duration HMM (Ken Chen, 2003)
+:[http://www.isle.uiuc.edu/pubs/2003/chen03interspeech.pdf Description], [{{SoftwarePath}}/HDK4_release source], [{{SoftwarePath}}/HDK4_release.tgz tgz]
-** bin.`arch` =  executes only on machines of type `arch`.  Type `arch` to see what machine you're on.
+==Signal Processing==
+;Singing-Voice Separation From Monaural Recordings Using Robust Principal Component Analysis (Po-Sen Huang, 2012)
+: [https://sites.google.com/site/singingvoiceseparationrpca/ Projects Description and Demo]  [http://www.ifp.illinois.edu/~huang146/RPCA_separation/RPCA_separation_codes.zip MATLAB code]
-** $PACKAGE_DIR/bin.Linux = packages with many binaries should remain in $PACKAGE_DIR, to avoid over-writing similarly-named programs in ../bin.Linux.
+;Nested STFTs (Dave Cohen, Camille Goudeseune, Mark Hasegawa-Johnson, 2009)
+:Efficient Simultaneous Multi-Scale Computation of FFTs
+: [http://fodava.gatech.edu/files/reports/GT-FODAVA-09-01.pdf Description], [http://zx81.isl.uiuc.edu/camilleg/gt-fodava-09-01-stft.c stft.c]
-(4) Change the installdir variable in your Makefile, according to your decision in part (3).  Type "make install" to install, then "make clean" to remove object files and such.
-=Phonetic Transcription Tool=
+;Improved Mistral (Qingsong Liu, 2009)
+:State of the Art Text-Independent Speaker Verification System,especially for NIST SRE
+:Based on [http://mistral.univ-avignon.fr/wiki/index.php/Main_Page Mistral Open Source package]
+:Improved and New Features:
+:* full factor analysis(eigenchannel and eigenvoice), instead of simple factor analysis(eigenchannel)
+:* multi-threads for Windows as well as Linux
+:* read HTK format feature/model
+:* effective Algorithm for fast implementation of FA.
+:* code optimization(for FA)
+:* fixed some bugs
+:Source: /ws/ifp-32-2/hasegawa/pineking/programs/Improved_Mistral
-This tool gives the American English phonetic transcription of any string.  It uses an HMM model to either generate a most likely phonetic transcription, or if a phonetic transcription is provided, it can perform forced alignment.  So, it gives a reasonable pronounciation for out-of-dictionary words, or partially pronounced words.
+;PVTK (Sarah Borys and Mark Hasegawa-Johnson, 2005-8)
+:[{{SoftwarePath}}/PVTK source], [{{SoftwarePath}}/PVTK.tgz tgz]
+:Extract HTK features as training vectors for libSVM, apply trained SVMs directly to feature files
-[[Phonetic Transcription Tool]].  You can try out the [http://mickey.ifp.uiuc.edu/speech/webpronounce/webpronounce.cgi demo] here.
+;VAD (Bowon Lee, 2007)
+:[http://www.isle.uiuc.edu/pubs/2007/lee07dspincars.pdf Description], [{{SoftwarePath}}/lee_vad source], [{{SoftwarePath}}/lee_vad.tgz tgz]
+:Voice activity detector with improved noise model
-=Parallel Computing=
+==Computation==
-==Sun Grid Engine on ifp-32==
+;Matlab GMM (Arthur Kantor, 2010)
-Bowon's brief introduction about the SGE [http://www.ifp.uiuc.edu/~bowonlee/research/cluster/linux_cluster.htm here]
+: [{{SoftwarePath}}/gmm source], [{{SoftwarePath}}/gmm.tgz tgz]
-Detailed SGE document, including Job Dependency[http://wikis.sun.com/display/GridEngine/Submitting+Extended+Jobs+and+Advanced+Jobs#SubmittingExtendedJobsandAdvancedJobs-ExampleExtendedJobExample]
+: A somewhat optimized Matlab toolbox for calculating the likelihood of many observations against many gaussian mixtures, each with many diagonal-covariance componenents.
-==MPI==
+;GMTK Parallel (Arthur Kantor, 2008)
-[http://search.cpan.org/~ajgough/Parallel-MPI-Simple-0.03/ Perl MPI Simple]
+:[[GMTK parallel tools|Description]]
+:Run GMTK commands in parallel on a compute cluster.  Email [[User:Arthur|Arthur]] for code.
-==Perl scripts for parallel processing of HTK commands==
+;HTK Parallel (Bowon Lee, 2006)
+:[http://www.ifp.uiuc.edu/~bowonlee/research/cluster/HTK_parallel.htm description], [{{SoftwarePath}}/HTK_parallel source], [{{SoftwarePath}}/HTK_parallel.tgz tgz]
+:These Perl scripts split an HTK command for parallel excution on an [http://www.ifp.uiuc.edu/~bowonlee/research/cluster/linux_cluster.htm SGE] cluster.
-Perl scripts for parallel processing of four HTK commands (HCopy, HERest, HVite, and HResults) are available.
+==Data==
+;Timeliner (Camille Goudeseune, 2012)
+:[http://zx81.isl.uiuc.edu/tmp/acmmm.pdf Description], [{{SoftwarePath}}/timeliner-2012-07-24.tgz Linux source tgz]
+:Browser for long audio files, with generated spectrograms and other derived features.
-. [http://www.ifp.uiuc.edu/~bowonlee/htk-pl/HCopy.pl HCopy.pl]
+;Matlab pfile I/O toolbox (Arthur Kantor, 2010)
+: [{{SoftwarePath}}/pfile source], [{{SoftwarePath}}/pfile.tgz tgz]
+: A Matlab toolbox for reading and writing ICSI pfile data format used by [http://ssli.ee.washington.edu/~bilmes/gmtk/ GMTK] and [http://www.icsi.berkeley.edu/Speech/qn.html QuickNet].
+: The toolbox is designed to work with large pfiles (hundreds of GB).
+: It is based on the [http://www.ee.columbia.edu/~dpwe/muscontent/matlab/pfread.m pfread.m] and [http://www.ee.columbia.edu/~dpwe/muscontent/matlab/pfinfo.m pfinfo.m] scripts by [http://www.ee.columbia.edu/~dpwe Dan Ellis].
-. [http://www.ifp.uiuc.edu/~bowonlee/htk-pl/HERest.pl HERest.pl]
+;Python library for reading/writing GMTK parameter files (Arthur Kantor, 2010)
+: [{{SoftwarePath}}/gmtkParam source], [{{SoftwarePath}}/gmtkParam.tgz tgz]
+: The library can read/write complete TrainableParameters files, as well as decision trees and most other objects allowed in GMTK parameter files.
+: This library is based on the code from [http://code.google.com/p/extended-hidden-vector-state-parser/ EHVS parser] project, and so is available under the [http://www.gnu.org/licenses/gpl-3.0.html GPL3] license.
-. [http://www.ifp.uiuc.edu/~bowonlee/htk-pl/HVite.pl HVite.pl]
+;Improved MVA (Arthur Kantor, 2008)
+:[http://mickey.ifp.uiuc.edu/speech/akantor/fisher/programs/bin.Linux/MVA Linux binary]
+:Perform mean and variance normalization and ARMA filtering
+:It's essentially [http://ssli.ee.washington.edu/people/chiaping/mva.html this] version but with
+:* better error reporting (e.g. failing to open file tells you so instead of core dumping)
+:* more accurate mean and variance estimation (doubles instead of floats in strategic places)
+:* faster computation in the case of MV (ARMA order 0)
+:svn location is svn://mickey.ifp.uiuc.edu/corporaNormalizationScripts/fisher/MVA.cc
-. [http://www.ifp.uiuc.edu/~bowonlee/htk-pl/HResults.pl HResults.pl]
+;DTMFseg (Bowon Lee, 2006)
+:[{{SoftwarePath}}/dtmfseg source], [{{SoftwarePath}}/dtmfseg.tgz tgz]
+:Segment audio files at DTMF tones
-Those scripts use the SGE (Sun Grid Engine) for job queuing.
+;Transcription tools (Mark Hasegawa-Johnson, 2005)
+:[{{SoftwarePath}}/transcription_tools source], [{{SoftwarePath}}/transcription_tools.tgz tgz]
+:Convert transcription formats
-Detailed information about the above Perl scripts can be found [http://www.ifp.uiuc.edu/~bowonlee/research/cluster/HTK_parallel.htm here].
+;Speechfileformats (Mark Hasegawa-Johnson, 2004)
+:[{{SoftwarePath}}/speechfileformats source], [{{SoftwarePath}}/speechfileformats.tgz tgz]
+:Read and write HTK files in matlab
-Brief introduction about the SGE can be found [http://www.ifp.uiuc.edu/~bowonlee/research/cluster/linux_cluster.htm here]
+;CTMRedit (Jul Cha and Mark Hasegawa-Johnson, 1999)
+:[http://www.isle.uiuc.edu/pubs/1990s/hasegawa-johnson99embs.pdf Description], [{{SoftwarePath}}/CTMRedit source], [{{SoftwarePath}}/CTMRedit.tgz tgz]
+:Manually and automatically segment CT and MR image stacks
+==LaTeX tools==
-Written by Bowon Lee, 02/24/2006
+[[LaTeX]] scripts to import figures from dia and pdf. (Arthur Kantor, 2010)
-== Parallel GMTK ==
+==Miscellaneous==
-http://mickey.ifp.uiuc.edu/speechWiki/index.php/GMTK_parallel_tools
+Other scripts written in perl, python, bash, and ruby can be found in [svn://mickey.ifp.uiuc.edu/scripts SVN archive].
-= More software written at ISLE =
+There is also [[Scripts Documentation| auto-generated documentation]] for them.
-* Acoustic Model Training
-** [http://www.isle.uiuc.edu/software/HDK4.tar.gz HDK]: HTK-based tools for training and testing explicit-duration hidden Markov models.  Contains makefiles and Windows binaries for the tools HDInit, HDrest, HDERest, HDHed,and HDVite.  Ken Chen, Last updated 5/2004.
-* File Format Tools
-** [http://www.isle.uiuc.edu/software/PVTK2005May23.tgz PVTK] Apply linear transforms to HTK files, or extract vectors in libSVM format.  Mark Hasegawa-Johnson, Last updated 5/2005.
-** [http://www.isle.uiuc.edu/software/transcription_tools2005May.tgz Transcription tools]: Perl scripts for manipulating speech database transcriptions and word lattices. Mark Hasegawa-Johnson, last updated 5/2005.
-** [http://www.isle.uiuc.edu/software/speechfileformats.tgz speechfileformats] Matlab tools for reading and writing HTK, MLF, and NIST files.
-= SVN =
-On linux, the client is svn, and should be installed everywhere.  On
-windows, download toroisesvn (http://tortoisesvn.tigris.org/)
-http://artis.imag.fr/~Xavier.Decoret/resources/svn/index.html

Software

From SpeechWiki

Latest revision as of 17:19, 4 February 2015

Contents

Statistical Speech Technology Group Software

Learning

Signal Processing

Computation

Data

LaTeX tools

Miscellaneous

Views

Personal tools

Navigation

Toolbox

Search