- The added line is THIS COLOR.
- The deleted line is THIS COLOR.
[[labs.beatcraft.com]] ~
[[CUDA]]~
#contents
* CUDA5/CentOS6.4 [#uc1c5ea8]
>
This article explains how to install CentOS 6.4 and CUDA5 on a PC, which is equipped with Tesla K20.~
~
#ref(CUDA_Tesla.jpg,,60%) ~
#ref(CUDA_Tesla.jpg,,45%) ~
~
* Hardware Specification [#de8b31c5]
>
The specification of hardware is shown below~
~
- CPU: Core i7 3770(3.4GHz,4core/8thread)
- Memory: 32GB(DDR3-12800 8GBx4)
- HDD: 1TB(SATA,7200rpm)
- GPU: ETSK20-5GER(nvidia Tesla K20c) ~
>
#ref(CUDA_PC.jpg,,45%) ~
~
#ref(CUDA_PC.jpg,,60%) ~
~
Since ETSK20-5GER does not have terminals of display outputs, such as D-SUB, DVI, HDMI, and so on, use the terminals of the mother board are used. To use the terminals, at the configuration of UEFI, the Primary Display is On Board.~
* To Install CentOS 6.4 (64bit) [#z1f9e64b]
>
The settings of installing CentOS 6.4 are shown below. ~
- Language & Keyboard: Japanese
- Package: Standard Desktop Install
- HDD Partition: Using whole disk, the standard configuration
- Network: DHCP
本稿では個別のパッケージのカスタマイズ設定は行っていません。~
またFirewall、SELinuxは共にDisableに設定しています。~
>
To install CentOS, there is no customization (modification) on packages.~
Please follow the instruction that the installer of CnetOS provides.~
Both Firewall and SELinux are configured as Disable.~
* CentOS インストール後設定 [#db8c96a0]
CentOS 6.4(64bit版)インストール後の設定について記載します。 ~
* Configuration for Post-Installation of CentOS[#db8c96a0]
>
This part describes how to prepare for installing CUDA after the installation of CentOS is completed.~
** CentOSのアップデート [#sf692f14]
インストール直後のKernelバージョンは以下のとおりです。~
** To update CentOS[#sf692f14]
>
The Kernel version of CentOS, which has been just installed, is shown below.~
>
$ uname -a
Linux localhost.localdomain 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
yum コマンドでアップデートを実行します。 ~
>
To update Kernel of CentOS, please apply to yum command.~
>
# yum update
アップデート完了後再起動を実行しアップデートしたKernelで起動することを確認します。~
>
After updating Kernel is completed, please reboot the OS and make sure that the updated Kernel works.~
>
# reboot
再起動後のKernelバージョンは以下のとおりです。~
>
After rebooting the CnetOS, please check the newer version of Kernel is installed. The newer version of Kernel is shown below.~
>
$ uname -a
Linux localhost.localdomain 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
beatユーザを作成し、sudoにbeatユーザを追加します。~
>
Create a new user. Then, add the new user to sudo account.~
>
# useradd beat
# passwd beat
>
# /usr/sbin/visudo
...<略>
...<Omitted>
## user MACHINE=COMMANDS
##
## The COMMANDS section may have other options added to it.
##
## Allow root to run any commands anywhere
root ALL=(ALL) ALL
beat ALL=(ALL) ALL
** パッケージ追加と設定 [#i1de06fb]
CUDAのインストールには直接関係はありませんが、作業をしやすくするため ~
ntpdとopensshをインストールします。 ~
** To add packages and to modify the configuration [#i1de06fb]
>
The addition of packages and modification of configuration are not directly related to install CUDA.~
However, to make installing process easier, install ntpd and openssh.~
~
ntpdをインストールします。~
Install ntpd.~
>
# yum install ntp
ntpサーバを追加します。 ~
>
Add ntp server.~
>
# vi /etc/ntp/step-tickers
# List of servers used for initial synchronization.
ntp.nict.jp
ntp.jst.mfeed.ad.jp
ntp.ring.gr.jp
ntpdを起動します。~
>
Boot ntpd.~
>
# /etc/init.d/ntpd start
ntpdをサービスに追加します。 ~
>
ntpd is added to the service.~
>
# chkconfig --level 3 ntpd on
# chkconfig --level 5 ntpd on
opensshをインストールします。~
既にインストールされている場合はこの手順はとばしてください。~
>
Install openssh.~
If openssh is already installed, please skip this part of the instruction.~
>
# yum install openssh
/etc/ssh/sshd_config内のPermitRootLoginと~
PermitEmptyPasswordsをnoに設定します。~
>
PermitRootLogin and PermitEmptyPasswords, which are located at the inside of the directory /etc/ssh/sshd_config, are set as No.~
>
# vi /etc/ssh/sshd_config
... <略>
... <Omitted>
# Authentication:
#LoginGraceTime 2m
PermitRootLogin no
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10
... <略>
... <Omitted>
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
PermitEmptyPasswords no
PasswordAuthentication yes
...
sshdを起動します。 ~
>
Boot sshd. ~
>
# /etc/init.d/sshd start
sshdをサービスに追加します。
>
Add service to sshd.~
>
# chkconfig --level 5 sshd on
# chkconfig --level 3 sshd on
* Tesla K20cドライバインストール [#n811fcbc]
Tesla K20cドライバインストールについて記載します。~
* To Install the Driver for Tesla K20c[#n811fcbc]
>
This section describes how to install the driver for Tesla K20c.~
** Tesla K20cドライバインストール準備 [#k7d204e8]
Tesla K20cドライバインストールのためにパッケージの追加、設定を行います。~
開発用パッケージ、ライブラリ群(Development tools)をインストールします。~
** Preparation for installing the Tesla K20c driver [#k7d204e8]
>
To install the deriver for Tesla K20c, the Developer Tools and packages of CentOS are installed. Then, these packages are configured if they are necessary.~
>
# yum groupinstall 'Development tools'
Tesla K20cのドライバ、CUDA5のインストール時にXが起動していると ~
エラーが起こるため、runlevel 3に変更し再起動を行います。 ~
>
An error will occur if X is booted during the diver for Tesla K20c and CUDA5, so runlevel is changed to runlevel 3. Then, it is rebooted.~
>
# vi /etc/inittab
...<略>
...<Omitted>
# Default runlevel. The runlevels used are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:3:initdefault:
# reboot
** Tesla K20cドライバのダウンロード [#w3abdcaf]
再起動後、以下のURLよりNVIDIA-Linux-x86_64-319.23.runをダウンロードします。~
** To download and install Tesla K20c driver[#w3abdcaf]
>
After rebooting, NVIDIA-Linux-x86_64-319.23.run is downloaded from the URL shown below.~
http://www.nvidia.co.jp/object/linux-display-amd64-319.23-driver-jp.html ~
~
NVIDIA-Linux-x86_64-319.23.runを実行し、ライセンスに同意後インストールが完了します。~
インストール完了後/proc/driver/nvidia/にversionファイルが作成されます。~
Execute NVIDIA-Linux-x86_64-319.23.run, and if you agree to the license agreement, then the instillation process of the driver for Tesla K20c is completed. After the installation is finished, the version file is created at the directory of /proc/driver/nvidea/.~
>
# chmod +x NVIDIA-Linux-x86_64-319.23.run
# ./NVIDIA-Linux-x86_64-319.23.run
# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 304.54 Sat Sep 29 00:05:49 PDT 2012
GCC version: gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
以下のURLに記載のあるように起動時ドライバを読み込むためのファイル(/etc/init.d/nvidia)を作成し、~
runlevel3,5の起動時に実行するよう設定します。~
>
Caution:~
The driver is needed to be re-built after the kernel update and reboot process are done.~
The re-build process is required since the updated kernel cannot read the nvidea driver. Without updating the driver, the CUDA5 Samples are not be executed.~
For the reference, the version of the new driver and the directory for the new nvidia driver are listed below.~
>
# uname -a
Linux tesla.beatcraft.com 2.6.32-358.18.1.el6.x86_64 #1 SMP Wed Aug 28 17:19:38 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 319.23 Thu May 16 19:36:02 PDT 2013
GCC version: gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
>
To read the driver, create a file at /etc/init.d/nvidea, and let runlevel3 and 5 be executed as the system is booted. For more details of the startup script, please visit the URL shown below.~
http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/index.html#install-cuda-software ~
>
# vi /etc/init.d/nvidia
#!/bin/bash
/sbin/modprobe nvidia
if [ "$?" -eq 0 ]; then
# Count the number of NVIDIA controllers found.
NVDEVS=`lspci | grep -i NVIDIA`
N3D=`echo "$NVDEVS" | grep "3D controller" | wc -l`
NVGA=`echo "$NVDEVS" | grep "VGA compatible controller" | wc -l`
N=`expr $N3D + $NVGA - 1`
for i in `seq 0 $N`; do
mknod -m 666 /dev/nvidia$i c 195 $i
done
mknod -m 666 /dev/nvidiactl c 195 255
else
exit 1
fi
# cd /etc/rc5.d/
# ln -s S99nvidia ../init.d/nvidia
# cd /etc/rc3.d/
# ln -s S99nvidia ../init.d/nvidia
* CUDA5のインストールとSampleの実行 [#gbfa8ff5]
CUDA5のインストールとCUDA5のSamplesについて記載します。~
* To Install CUDA5 and Execute Samples [#gbfa8ff5]
>
This section shows how to install CUDA5 and executes a sample CUDA program.~
** CUDA5のダウンロード [#ge140c64]
CentOS6 64bit版CUDA(cuda_5.0.35_linux_64_rhel6.x-1.run)を以下のURLから ~
ダウンロードします。~
** To download CUDA5 [#ge140c64]
>
Download CUDA for CentOS6 64bit version (cuda_5.0.35_linux_64_rhel6.x-1.run) from URL below.~
https://developer.nvidia.com/cuda-downloads ~
~
CUDA5のSampleインストール時にPATHが見つからずSampleがインストールされないのを防ぐため、~
.bashrcに予めPATHを追加します。 ~
Add a PATH for CUDA5 in .bashrc. This is a precautionary measure. If the sample programs can not find the PATH, the samples cannot be installed correctly. ~
>
$ cd /home/beat
$ vi .bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
export PATH=/usr/local/cuda-5.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-5.0/lib64:/usr/local/cuda-5.0/lib:$LD_LIBRARY_PATH
# source .bashrc
** パッケージのインストール [#md50346c]
Samplesのビルドで必要な以下のパッケージを予めインストールします。~
** To install packages [#md50346c]
>
To build the samples, these packages listed below are required.~
>
# yum install freeglut-devel.x86_64 libXi-devel.x86_64 mpich2-devel.x86_64 libXmu-devel.x86_64
cuda_5.0.35_linux_64_rhel6.x-1.run を実行します。~
インストール先ディレクトリはデフォルトの/usr/local/cuda-5.0を指定します。~
>
Execute cuda_5.0.35_linux_64_rhel6.x-1.run. Specify the directory that the sample program is installed at the default directory, which is /usr/local/cuda-05.~
>
# chmod +x cuda_5.0.35_linux_64_rhel6.x-1.run
# ./cuda_5.0.35_linux_64_rhel6.x-1.run
Samplesはデフォルトで/usr/local/cuda-5.0/samplesと/root/NVIDIA_CUDA-5.0_Samplesに ~
インストールされます。~
>
Sample programs are installed at the directories of /usr/local/cuda-5.0/samples and /root/NVUDA-5.0_Samples. ~
~
rootユーザでビルドできるようにld.so.conf.d/cuda.confを追加しldconfigを実行します。~
To let a root user build the samples, add ld.so.conf.d/cuda.conf, then, execute ldconfig.~
>
# cat /etc/ld.so.conf.d/cuda.conf
/usr/local/cuda-5.0/lib
/usr/local/cuda-5.0/lib64
# ldconfig
** CUDA5 Samplesのビルドと実行 [#v02860fe]
NVIDIA_CUDA-5.0_Samplesをbeatユーザでビルド、実行できるようにします。~
/root/NVIDIA_CUDA-5.0_Samplesからコピーし、オーナーをbeatに変更しmakeを実行します。~
** To build and execute the samples of CUDA5 [#v02860fe]
>
Let a user build and execute NVIDIA_CUDA-5.0_Samples.~
Copy the file NVIDIA_CUDA-5.0_Samples from /root/NVIDIA_CUDA-5.0_Samples.
Change the owner of the file NVIDIA_CUDA-5.0_Samples to a user. Then, execute make.~
>
# cp -afr /root/NVIDIA_CUDA-5.0_Samples /home/beat/.
# cd /home/beat
# chown -R beat.beat NVIDIA_CUDA-5.0_Samples
$ cd ~/NVIDIA_CUDA-5.0_Samples
$ make
ビルド完了後 NVIDIA_CUDA-5.0_Samples/bin/linux/release/に移動し、~
NVIDIA CUDA Getting Started Guide for Linuxのページ内Running the Binariesに ~
記載されているdeviceQuery、bandwidthTestを実行します。 ~
deviceQuery、bandwidthTest共にTesla K20cを用いていることが確認できます。~
>
After the build process is completed, Change the directory to /NVIDIA_CUDA-5.0_Samples/bin/linux/release/. Then, execute sample programs, which are listed on Running the Binaries of the NVIDIA CUDA Getting Started Guide for Linux. The URL of this page is show below. The names of sample programs, which are going to be executed are deviceQuery and bandwidthTest.~
~
http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/index.html#running-binaries ~
~
To look at the outputs shown below, both deviceQuery and bandwidthTest recognize Tesla K20c.~
>
$ cd ~/NVIDIA_CUDA-5.0_Samples/bin/linux/release/
$ ./deviceQuery
./deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
Device 0: "Tesla K20c"
CUDA Driver Version / Runtime Version 5.0 / 5.0
CUDA Capability Major/Minor version number: 3.5
Total amount of global memory: 4800 MBytes (5032706048 bytes)
(13) Multiprocessors x (192) CUDA Cores/MP: 2496 CUDA Cores
GPU Clock rate: 706 MHz (0.71 GHz)
Memory Clock rate: 2600 Mhz
Memory Bus Width: 320-bit
L2 Cache Size: 1310720 bytes
Max Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536,65536), 3D=(4096,4096,4096)
Max Layered Texture Size (dim) x layers 1D=(16384) x 2048, 2D=(16384,16384) x 2048
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Maximum sizes of each dimension of a block: 1024 x 1024 x 64
Maximum sizes of each dimension of a grid: 2147483647 x 65535 x 65535
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Enabled
Device supports Unified Addressing (UVA): Yes
Device PCI Bus ID / PCI location ID: 1 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 5.0, CUDA Runtime Version = 5.0, NumDevs = 1, Device0 = Tesla K20c
>
$ ./bandwidthTest
[CUDA Bandwidth Test] - Starting...
Running on...
Device 0: Tesla K20c
Quick Mode
Host to Device Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(MB/s)
33554432 6431.7
Device to Host Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(MB/s)
33554432 6396.1
Device to Device Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(MB/s)
33554432 143752.5
** 設定完了後のHDD使用容量 [#c97ef8c4]
設定完了後のHDDの使用容量は以下のとおりです。~
** HDD space[#c97ef8c4]
>
After the install and configuration of CUDA, the remaining Hard Disk space is shown below.~
>
# df -h
Filesystem Size Used Avail Use% マウント位置
/dev/mapper/vg_beattesla-lv_root
50G 4.3G 45G 9% /
tmpfs 16G 0 16G 0% /dev/shm
/dev/sda1 485M 66M 394M 15% /boot
/dev/mapper/vg_beattesla-lv_home
852G 1.4G 808G 1% /home
* 更新履歴 [#cbed8220]
2013/05/31 初稿掲載 ~
2013/06/01 文言修正 ~
* Revision History [#cbed8220]
>
-2013/06/04 This article is initially uploaded.~
-2013/09/26 The tittle of the section is changed from "To download Tesla K20c driver" to "[[To download and install Tesla K20c driver>http://labs.beatcraft.com/en/index.php?CUDA5%2FCentOS6.4#w3abdcaf]]." Caution is added at the section.~