Subject: Re: SVM training From: "Sarah E. Borys" Date: Mon, 1 May 2006 08:32:19 -0500 (CDT) To: HJ CC: ocetin@icsi.berkeley.edu, klivescu@csail.mit.edu There is actually a step 3.5 in between steps 3 and 4 and that is to make reduced set SVMs [1]. The idea is that the reduced set svm has less vectors and will take much less computation time. The first step is to download the statistical pattern recognition toolbox for matlab. I believe it is available at http://cmp.felk.cvut.cz/~xfrancv/stprtool/ The function you will need to use is calles rsrbf. As you can guess from the function name, this method only works for rbf svms. If you have any other type of kernel, you will not be avle to use the matlab toolbox. The svm needs to be in a specific form for the toolbox to be able to use it. It needs to be in a structure, which I will call s, and it needs to have 3 components: s.sv, s.Alpha, s.options. s.sv is an NxV matrix that contains your support vectors, where N is the length of the support vector and V is the number of support vectors. s.Alpha is Vx1 vector of all the alpha values that correspond to each support vector. s.options has 2 components: s.options.ker and s.options.arg. s.options.ker should be set to 'rbf' or the code will not run. s.options.arg should be equal to sqrt(.5/gamma). This is because matlab actually treats the kernel as a gaussian kernel and not an rbf. Once you have your svm in this form (I use a perl script to do it), the function you need to use is rsrbf. The usage is simple: rs = rsrbf(s, struct('nsv', round(length(s.Alpha)/10))); where rs will be the reduced set svm and s is the original svm. The second arguement tells rsrbf how many reduced set vectors you want. Right now, it is says to reduce the number of vectors by a factor of 10. I think the default if you don't include the second arguement is 1/2 the original number of vectors. Once you have the reduced set svm, you will need to find a way to print it and copy the header (adjusting the number of support vectors the header specifies if need be), and you can quickly run your new reduced set svm on your data to see how well it does. I usually try not to let the performance degrade more then 1%. I usually reduce the support vectors by a factor of 10. A lot of the time, I get lucky and can reduce the number of support vectors by a factor of 100 without suffering too much in accuracy. You may have to play around a bit. Sarah [1] C.J.C. Burges, "Simplified Support Vector Decision Rules", 13th International Conference on Machine Learning, 1996 On Fri, 28 Apr 2006 jhasegaw@uiuc.edu wrote: > > Hi Ozgur, > > > > Sorry I haven't sent you SVM code yet. I am currently > > rewriting PVTK from scratch, so the old version is kind of > > halfway supported -- but Sarah uses it here at Illinois, and I > > will support it for both of you as well as I can. > > > > Sarah, would you please read through this to see if I've > > missed anything important? Thank you!! > > > > The process we currently use is as follows: > > > > (1) Create HTK-style master label files (MLF) containing the > > start time and end time of each phonetic segment in each file > > in your training data. I use perl. There is some perl code > > here for things like timit2mlf: > > http://www.isle.uiuc.edu/software/transcription_toolsMay2005.tgz, > > but it won't do SRI lattice to MLF. > > > > (2) Extract the training vectors, using the program VExtract. > > This is part of PVTK: http://www.isle.uiuc.edu/software/, > > choose the link "PVTK" to download source. I have attached a > > linux binary. In order to compile PVTK, you need to have the > > static HTK libraries compiled; edit the Makefile to tell gcc > > where they are. > > > > The syntax for VExtract is something like: > > > > VExtract -T 1 -c /+1/^(m|n|ng)/ -c /-1/^[^mn]/ -h /-4:2:4/ -m > > 5000,corpus,locked -f train.stats -o train.toks -I ws97.mlf -S > > trainingfiles.scp > > > > This command means that every line in the MLF whose third > > column matches the regex ^(m|n|ng) is a segment whose frames > > are examples of class +1. Every line matching ^[^mn] is a > > segment whose frames are examples of class -1. From these > > segments, training tokens will be extracted. A training token > > is constructed by concatenating some number of frames from the > > input parameter files, as specified by the -h option: in this > > case, each extracted training token is the concatenation of an > > input frame, the frames 4 and 2 time steps prior, and the > > frames 2 and 4 time steps after. Up to 5000 example frames > > from each class will be extracted. These frames will be > > extracted at points uniformly spaced throughout the entire > > corpus, i.e., the +1 extractor will see the corpus as if, > > after the spectral concatenation specified by -h were > > performed, then all files were concatenated end-to-end, and > > then all frames not matching the +1 regex had been removed; > > the removing frames are decimated to find 5000 frames of the > > +1 class. These vectors are normalized so that each dimension > > has zero mean and unit standard deviation; the original mean > > and standard deviation are stored in the file train.stats. > > The resulting normalized vectors are saved out in plain text > > svmlight format to train.toks. > > > > (3) The actual SVM training is still performed using svmlight > > or libSVM. In fact, I recommend svmlight, because I don't > > know how to get real-valued discriminants out from libSVM -- > > Sarah might. > > > > (4) Once you've trained the SVMs, you can apply it to every > > frame in an input speech file using the VApplySVMs program. > > > > - Mark > >