[[labs.beatcraft.com]]
[[Deep Learning]]
#Contents
*Overview [#u3eb53fe]
>
This article explains how to install [[Pylearn2>http://deeplearning.net/software/pylearn2/]] on CUDA. Pylearn2 is a machine learning library for Python. Since most of functionality of Pylearn2 is built upon the top of Theano, the models and algorithms written in Pylearn2 are expressed in mathematical expressions. Theano compiles these models and algorithms to CUDA.~
~
This article uses the same hardware, which is used for explaining how to install CUDA on Ubuntu in the article of [[CUDA6.5/Ubuntu 14.04]]. The system is equipped with Tesla K20c, and CUDA tools 6.5 is installed on Ubuntu 14.04. This article shows how to set up Pylearn2 on CUDA 6.5.
*Setting up Pylearn2 [#i37fce6c]
** Installing Ubuntu 14.04 and Configuring CUDA [#x720f1ec]
>
To enable Pylearn2 to work on GPU at backend correctly, you have to set up CUDA correctly on the OS of the system. Please check this article for installing CUDA Toolkit 6.5 on Ubuntu 14.04.~
*** Python Modules [#l95b7946]
>
If the Python modules, which are required for installing Pylearn2 and Theano, are available at repository, please use the command line ''apt-get install''. Otherwise use pip to install the modules.~
** Installing Theano [#e404e72c]
>
First, the required libraries, which are needed for installing Theano, are installed. The [[list of requirements>http://deeplearning.net/software/theano/install.html#install]] is shown is shown below.~
~
- Python 2.6 or greater (This is the default Python on Ubuntu 14.04.)
- g++
- python-dev
- Numpy 1.5.0 or greater
- SciPy
- BLAS (Basic Liner Algebra Subprograms, Level3 function is required)
~
Then, the following optional packages are also installed.~
~
- node
- Sphinx 0.5.1 or greater
- Git
- pydot
- CUDA (already installed)
- libgpuarray
>
For the reference, please read the article listed below. It lists the crucial points for installing Theano on Ubuntu. [[Easy Installation of an Optimized Theano on Current Ubuntu>http://deeplearning.net/software/theano/install_ubuntu.html]].~
>
Install git and Python modules by applying apt-get.~
$ sudo apt-get install git python-dev paython-numpy python-scipy python-pip python-nose python-sphinx python-pydot
>
To install BLAS, use OpenBLAS. The instructions of how to install openBLAS are listed at [[this page>http://deeplearning.net/software/theano/install_ubuntu.html#manual-openblas-instruction]]. Please follow the instructions. (The package of OpenBLAS for Ubuntu puts a limit on the number of threads at two. Therefore, please build and install OpenBLAS from the scratch.)
$ sudo apt-get install gfortran
$ git clone git://github.com/xianyi/OpenBLAS
$ cd OpenBLAS
$ make FC=gfortran
$ sudo make PREFIX = /usr/local install
$ sudo idconfig
>
Before installing libgpuarray, please install the requirements for libgpuarray, first. For the instructions of how to install these requirements, please visit [[this page>http://deeplearning.net/software/libgpuarray/installation.html]].~
$ sudo apt-get install cmake check python-mako cython
>
Obtaining the source code of libbgpuarray from git.
$ git clone https://github.com/Theano/libgpuarray.git
$ cd libgpuarray
>
If you try to execute the build command of Cmake, an error will occur at the link to pthread.~
To avoid this error to happen, CMakeLists.text will be modified in the way as it is shown below.
$ cd src
$ vim CMakeLists.text
~
Before modifying the CMakeLists.text
if(CUDA_FOUND)
target_link_libraries(pthread $ {CUDADRV_LIBRARY} $ {CUDA_CUBLAS_LIBRARIES})
target_link_libraries(gpuarray-static $ {CUDADRV_LIBRARY} $ {CUDA_CUBLAS_LIBRARY})
endif()
　↓
After modifying the CMakeLists.text
if(CUDA_FOUND)
target_link_libraries(gpuarrat pthread $ {CUDADRV_LIBRARY} $ {CUDA_CUBLAS_LIBRARIES})
target_link_libraries(gpuarray-static pthread $ {CUDADRV_LIBRARY} $ {CUDA_CUBLAS_LIBRARY})
endif()
>
After the modification is completed, going back to the directory below libgpuarray. To follow the instructions below, build and install libgpuarray.
$ cd ..
$ mkdir Build
$ cd Build
$ cmake.. -DCMAKE_BUILD_TYPE=Release
$ make
$ sudo make install
$ sudo idconfig
$ cd ..
>
pygpu, which is included in libgpuarray, is installed by setup.py. To install pygpu, please aply the command lines listed below.~
$ python setup.py build
$ sudo python setup.py install
This is basically the end of installing the pre-requirements for the installation of Theano. Since Pylearn2 recommends to install a newer version of Theano ([[please look at this page>http://deeplearning.net/software/pylearn2/#download-and-installation]]), download and instal bleeding-edge version of Theano. To follow the instructions listed at the URL shown below, please git the newest version of Theano.~
[[http://deeplearning.net/software/theano/install.html#bleeding-edge-install-instructions]]
$ pip install -- update -- no-deps git + git://github.com/Theano/Theano.git
** Adjusting the configuration of Theano [#y649e64c]
>
As the installation of Theano is succeed, Theano works at the CPU backend. As the article at [[this page>http://deeplearning.net/software/theano/install.html#gpu-linux]], Theano is needed to be configured for using GPU.~
>
Create .theanorc at the home directory, write down the contents listed below.~
[global]
floatX=float32
device=gpu
[mode]=FAST_RUN)
[nvcc]
fastmath = True
[cuda]
root=/usr/local/cuda
[blas]
Idflags= -Iopenblas
>
Executing Theano, .theanorc is read. To examine whether GPU is effective or not, please execute an example, which is listed at [[this page>http://deeplearning.net/software/theano/tutorial/using_gpu.html#using-gpu]]. The other example of how to check the effectiveness of GPU is to execute check_blas.py changing the setting at device option Then, the outputs and the durations of executions are compared among different values at device option. This is how to execute check_blas.py.~
~
Not using Tesla K20c~
$ THEANO_FLAGS=floatX=float32,device=cpu python /usr/local/lib/python2.7/dist-packages/theano/misc/check_blas.py
-- Skipping --
mkl_info:
NOT AVAILABLE
Numpy dot module: numpy.core._dotblas
Numpy location: /usr/lib/python2.7/dist-packages/numpy/__init__.pyc
Numpy version: 1.8.2
We executed 10 calls to gemm with a and b matrices of shapes (2000, 2000) and (2000, 2000).
Total execution time: 1.09s on CPU (with direct Theano binding to blas).
Try to run this script a few times. Experience shows that the first time is not as fast as followings calls. The difference is not big, but consistent.
>
Using Tesla K20c~
$ THEANO_FLAGS=floatX=float32,device=gpu python /usr/local/lib/python2.7/dist-packages/theano/misc/check_blas.py
Using gpu device 0: Tesla K20c
-- Skipping --
mkl_info:
NOT AVAILABLE
Numpy dot module: numpy.core._dotblas
Numpy location: /usr/lib/python2.7/dist-packages/numpy/__init__.pyc
Numpy version: 1.8.2
nvcc version:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2014 NVIDIA Corporation
Built on Thu_Jul_17_21:41:27_CDT_2014
Cuda compilation tools, release 6.5, V6.5.12
We executed 10 calls to gemm with a and b matrices of shapes (2000, 2000) and (2000, 2000).
Total execution time: 0.08s on GPU.
Try to run this script a few times. Experience shows that the first time is not as fast as followings calls. The difference is not big, but consistent.
** Installing Pylearn2 [#p9a3fd11]
>
To install Pylearn2, PyYAML and PIL are required besides installing Theano.~
(Because PIL is a dependent of CUDA, PIL is installed when CUDA is introduced to the system.)~
$ sudo apt-get install python-yamal pyathon-pil
>
As the pre-requirements for Pylearn2 are installed, finally the source code of Pylearn2 is downloaded. To download the code, do git clone for the source code. Then, install the code.~
$ git clone git://github.com/lisa-lab/paylearn2.git
$ cd pylearn2
sudo python setup.py.develop
>
After the installation process is completed, please add the configuration of Data Path, which is required for executing Pylearn2, to .bashrc.~
Please create the Data directory for store the data. Basically, you can create this directory anywhere as long as where your write permission is effective. In this example the Data directory is created under the Home directory and specified it to .bashrc.~
$ makedir -p pylearn2data
$ exho 'exporet PYLEARN2_DATA_PATH=/home/beat/pylearn2data >> .bashrc
$ .~/bashrc
>
Then, install matplotlib. This is required for executing a tutorial of Pylearn2.
$ sudo apt-get install
** Checking the Operation of Pylearn2 [#af8d8ec2]
>
To check whether Pylearn2 is set up correctly or not, execute Quick-start example. The details of this example is listed at [[this page>http://deeplearning.net/software/pylearn2/tutorial/index.html#tutorial]].~
$ cd /home/beat/work/pylearn2/pylearn2/scripts/tutorials/grbm_smd
$ python make_dataset.py
Using gpu device 0: Tesla K20c
Traceback (most recent call last):
File "make_dataset.py", line 27, in <module>
train = cifar10.CIFAR10(which_set="train")
File "/home/beat/work/pylearn2/pylearn2/datasets/cifar10.py", line 76, in __init__
raise IOError(fname + " was not found. You probably need to "
IOError: /home/beat/pylearn2/data/cifar10/cifar-10-batches-py/data_batch_1 was not found. You probably need to download the CIFAR-10 dataset by using the download script in pylearn2/scripts/datasets/download_cifar10.sh or manually from http://www.cs.utoronto.ca/~kriz/cifar.html
>
Before executing the example, please download the dataset. Otherwise, warnings will appear.~
$ cd　../../datasets
$ ./download_cifer10.sh
Downloading and unzipping CIFAR-10 dataset into /home/beat/pylearn2/data/cifar10...
cifar-10-batches-py/
cifar-10-batches-py/data_batch_4
cifar-10-batches-py/readme.html
cifar-10-batches-py/test_batch
cifar-10-batches-py/data_batch_3
cifar-10-batches-py/batches.meta
cifar-10-batches-py/data_batch_2
cifar-10-batches-py/data_batch_5
cifar-10-batches-py/data_batch_1
2015-01-16 15:39:45 URL:http://www.cs.utoronto.ca/~kriz/cifar-10-python.tar.gz [170498071/170498071] -> "-" [1]
>
As the download is completed, the example is re-executed.~
$ cd ../tutorials/grbm_smd/
$ python make_dataset.py
Using gpu device 0: Tesla K20c
loading file /home/beat/pylearn2/data/cifar10/cifar-10-batches-py/data_batch_1
loading file /home/beat/pylearn2/data/cifar10/cifar-10-batches-py/data_batch_2
loading file /home/beat/pylearn2/data/cifar10/cifar-10-batches-py/data_batch_3
loading file /home/beat/pylearn2/data/cifar10/cifar-10-batches-py/data_batch_4
loading file /home/beat/pylearn2/data/cifar10/cifar-10-batches-py/data_batch_5
loading file /home/beat/pylearn2/data/cifar10/cifar-10-batches-py/test_batch
/home/beat/work/pylearn2/pylearn2/datasets/preprocessing.py:1187: UserWarning: This ZCA preprocessor class is known to yield very different results on different platforms. If you plan to conduct experiments with this preprocessing on multiple machines, it is probably a good idea to do the preprocessing on a single machine and copy the preprocessed datasets to the others, rather than preprocessing the data independently in each location.
warnings.warn("This ZCA preprocessor class is known to yield very "
computing zca of a (150000, 192) matrix
cov estimate took 0.27054309845 seconds
eigh() took 0.0118489265442 seconds
/home/beat/work/pylearn2/pylearn2/datasets/preprocessing.py:1280: UserWarning: Implicitly converting mat from dtype=float64 to float32 for gpu
'%s for gpu' % (mat.dtype, floatX))
/home/beat/work/pylearn2/pylearn2/datasets/preprocessing.py:1283: UserWarning: Implicitly converting diag from dtype=float64 to float32 for gpu
'%s for gpu' % (diags.dtype, floatX))
>
To use the script, which is located at the directory of pylearn2/scripts/, set PATH to this directory.
$ export PATH=/home/beat/work/pylearn2/pylearn2/scripts:$PATH
>
Execute train.py~
$ train.py cifar_grbm_smd.yaml
Using gpu device 0: Tesla K20c
Parameter and initial learning rate summary:
W: 0.10000000149
bias_vis: 0.10000000149
bias_hid: 0.10000000149
sigma_driver: 0.10000000149
Compiling sgd_update...
Compiling sgd_update done. Time elapsed: 7.771741 seconds
compiling begin_record_entry...
compiling begin_record_entry done. Time elapsed: 0.089102 seconds
Monitored channels:
bias_hid_max
bias_hid_mean
bias_hid_min
bias_vis_max
bias_vis_mean
bias_vis_min
h_max
h_mean
h_min
learning_rate
objective
reconstruction_error
total_seconds_last_epoch
training_seconds_this_epoch
Compiling accum...
graph size: 91
Compiling accum done. Time elapsed: 0.814388 seconds
Monitoring step:
Epochs seen: 0
Batches seen: 0
Examples seen: 0
bias_hid_max: -2.00000023842
bias_hid_mean: -2.00000023842
bias_hid_min: -2.00000023842
bias_vis_max: 0.0
bias_vis_mean: 0.0
bias_vis_min: 0.0
h_max: 8.27688127174e-05
h_mean: 1.74318574864e-05
h_min: 9.55541054282e-06
learning_rate: 0.100000016391
objective: 14.4279642105
reconstruction_error: 70.9217071533
total_seconds_last_epoch: 0.0
training_seconds_this_epoch: 0.0
/home/beat/work/pylearn2/pylearn2/training_algorithms/sgd.py:586: UserWarning: The channel that has been chosen for monitoring is: objective.
str(self.channel_name) + '.')
Time this epoch: 25.525986 seconds
Monitoring step:
Epochs seen: 1
Batches seen: 30000
Examples seen: 150000
bias_hid_max: -0.257617294788
bias_hid_mean: -1.75261676311
bias_hid_min: -2.36502599716
bias_vis_max: 0.160428583622
bias_vis_mean: -0.00086586253019
bias_vis_min: -0.220651045442
h_max: 0.410839855671
h_mean: 0.0542325824499
h_min: 0.0116947097704
learning_rate: 0.100000016391
objective: 3.62195086479
reconstruction_error: 29.2136707306
total_seconds_last_epoch: 0.0
training_seconds_this_epoch: 25.5259819031
monitoring channel is objective
Saving to cifar_grbm_smd.pkl...
Saving to cifar_grbm_smd.pkl done. Time elapsed: 0.025346 seconds
Time this epoch: 25.384062 seconds
Monitoring step:
Epochs seen: 2
Batches seen: 60000
Examples seen: 300000
bias_hid_max: -0.305719166994
bias_hid_mean: -2.00991845131
bias_hid_min: -2.78829908371
bias_vis_max: 0.185681372881
bias_vis_mean: -0.000737291120458
bias_vis_min: -0.177558258176
h_max: 0.394594907761
h_mean: 0.0468980930746
h_min: 0.0104174567387
learning_rate: 0.100000016391
objective: 3.38024163246
reconstruction_error: 28.5441741943
total_seconds_last_epoch: 25.89610672
training_seconds_this_epoch: 25.3840618134
monitoring channel is objective
Saving to cifar_grbm_smd.pkl...
Saving to cifar_grbm_smd.pkl done. Time elapsed: 0.025256 seconds
Time this epoch: 25.465318 seconds
Monitoring step:
Epochs seen: 3
Batches seen: 90000
Examples seen: 450000
bias_hid_max: -0.302897870541
bias_hid_mean: -2.12691950798
bias_hid_min: -3.09918379784
bias_vis_max: 0.168909445405
bias_vis_mean: 0.000913446128834
bias_vis_min: -0.161776274443
h_max: 0.389986425638
h_mean: 0.0441780276597
h_min: 0.00789143983275
learning_rate: 0.100000016391
objective: 3.30141615868
reconstruction_error: 28.4002838135
total_seconds_last_epoch: 25.7539100647
training_seconds_this_epoch: 25.4653167725
monitoring channel is objective
Saving to cifar_grbm_smd.pkl...
Saving to cifar_grbm_smd.pkl done. Time elapsed: 0.025410 seconds
Time this epoch: 25.288767 seconds
Monitoring step:
Epochs seen: 4
Batches seen: 120000
Examples seen: 600000
bias_hid_max: -0.329535990953
bias_hid_mean: -2.19633841515
bias_hid_min: -3.181681633
bias_vis_max: 0.171140804887
bias_vis_mean: -0.000430780899478
bias_vis_min: -0.197250261903
h_max: 0.39044636488
h_mean: 0.0431808494031
h_min: 0.00783428177238
learning_rate: 0.100000016391
objective: 3.28094577789
reconstruction_error: 28.5033798218
total_seconds_last_epoch: 25.8351802826
training_seconds_this_epoch: 25.2887706757
monitoring channel is objective
growing learning rate to 0.101000
Saving to cifar_grbm_smd.pkl...
Saving to cifar_grbm_smd.pkl done. Time elapsed: 0.025562 seconds
Saving to cifar_grbm_smd.pkl...
Saving to cifar_grbm_smd.pkl done. Time elapsed: 0.025118 seconds
>
cifar_grbm is output.~
~
To check the results, apply show_weights.py and cifar_grbm_smd.pkl. If you execute it without any configuration, you will receive a warning for asking the configuration. Apply the instruction on the warning.
$ export PYLERN2_VIEWER_COMMAND=”Eog--new-instance”
Then, try to execute the example, again.~
$ show_weights.py cifar_grbm_smd.pkl
As the command line above is executed, a Gabor filter, which is generated from the learning experience of pylearn2, is displayed on Eye of Gnome.~
#ref(01.png,,100%) ~
>
Using --out, set up the options. Then, output the results on the image file. The command lines below indicate how to apply the options and show the outcomes.~
$ show_weights.py cifar_grbm_smd.pkl --out=weights.png
Using gpu device 0: Tesla K20c
making weights report
loading model
loading done
loading dataset...
...done
smallest enc weight magnitude: 3.91688871559e-07
mean enc weight magnitude: 0.0586505495012
max enc weight magnitude: 0.99245673418
min norm: 0.899496912956
mean norm: 1.37919783592
max norm: 1.96336913109
* Revision History [#f37ec3f2]
>
- 2015/02/13 This article is initially uploaded
![[PukiWiki]](image/bc_logo.png "[PukiWiki]"){kind=logo}
