2015年11月20日 星期五

Caffe - Deep Learning Tool Installation

 

Deep Learning 有幾種常用的 Tools

* Caffe based on C, but with Python interface: UC Berkeley 開發。Google, CEVA 使用。Mainly for image.

* Torch based on Lua (ANSI C): NYU 開發。Facebook AI and Google use it.

* Theano and Pylearn based on Python : University of Toronto

* TensorFlow: Google

* SystemML: IBM

 

本文主要 focus 在如何設定 Caffe.  

Caffe 幾個特點:

底層是 C (and C++?) 所以速度很快。不過可以用 python (even ipython) 做成 interface.  所以用起來容易。

Protobuf:  from Google, for easy and fast data structure.

LMDB: from SAMS, for lightning database

HDF5: from xxx, for database.

 

OS (Ubuntu 14.04 64bit)

我用 Microsoft surface pro 3 with windows 10.  不過為了方便 try-and-error, 我用 VMware workstation player 12 with guest OS:  Ubuntu 14.04 (64-bit)


Install Ubuntu on VMware workstation player 12

* First install VMware workstation player 12 (free)

* Ubuntu 部份:

update Ubuntu’s update --> Yes.

sudo apt-get install tcsh --> No need, just use bash

sudo apt-get install unit-tweak-tool (to increase terminal font size) --> No need, use system setting: displays: scale 1.5 instead!

sudo apt-get install samba  (1. edit /etc/smb-conf; 2. smbpasswd -a alu; 3. smb start)  -->  No need.

-> update Ubuntu 之後,重新 reboot 會有 could not apply the stored configuration for monitors 

Solution: 

Once logged in change your directory to .config folder in your home directory.

cd .config

Rename monitors.xml file to monitors.xml.bkp. In here, we are keeping a backup file just to play safe.

mv monitors.xml monitorx.xml.bkp

 

Ubuntu Installation

General dependencies

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler sudo apt-get install --no-install-recommends libboost-all-dev

CUDA:  download from Nvidia (only 64-bit).  如果沒有 GPU 可省略。

BLAS:  使用 ATLAS :  

sudo apt-get install libatlas-base-dev

Remaining dependencies 14.04:

sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

 

 

Python 部份最好後處理

 

Caffe Installation

Download from Github

* cp Makefile.config.sample Makefile.config

* Edit Makefile.config

Set 1:  CPU only, no python layer link.  不支持 pycaffe

make all;  make test; make runtest  ---> OK  (只要先不 install python)

 

 

Python (for pycaffe interface)

sudo apt-get install python-dev  ----> NONONONO!!!!

不過問題是沒有 numpy.  所以需要 install pip (python package management) and maybe not virtuenv (for coexist of python 2.7 and python 3.x) --> NO!

—> 最直接是只用 python 2.7 且 install Anaconda Python (similar to Enthought python), 不要做 sudo apt-get install python-dev! 

 

 

Linux Anaconda Installation

  1. Download the installer.
  2. After downloading the installer, in your terminal window execute for Python 2.7.10
    bash Anaconda2-2.4.0-Linux-x86_64.sh
    Anaconda2-2.4.0 是 2.7.10
    Anaconda-2.0.0 是 2.7.6  (OK, but need conda upgrade xxx) 

      bash Ana… .sh  to install anaconda python  

 

      Option A: 2.4.0 --> (python 2.7.10)   ====>  NOT GOOD !!!

      2.4.0 (python 2.7.10) in /home/alu/anaconda2  

      Back to caffe and uncomment python part --> make clean; make all; make test OK; but make runtest with error:

       libhdf5_hl.so.10 not found.  

 

 ==> Option B: 2.0.0 --> (python 2.7.6) in /home/alu/anaconda  ====>  GOOD 

      No error 

 

      >  condo update conda

      > pip install protobuf

 

     > make pycaffe

 

 

How to Use or Verify Python?

Use ipython notebook (in caffe-master directory)

 

 

 

 

Install boost for python!??  no

 

Error when running ipython notebook

No module name google.protobuf.internal

> pip install protobuf

 

 

> make all; make test ok. but make runtest failed.  no libhdf5_hl.so.10 not found.  只有在 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so.7  (這是在 general dependencies 時 install libhdf5-serial-dev 時 Ubuntu 的 default python 是 2.7.6 (?).  但 Anaconda python 是 python 2.7.10.

  Ubuntu built-in python lib seems to be out-of-date.   Solution:

  - install Anaconda python package, and check the location of libhdf5_hl.so.10  一般是在 ~/anaconda/lib

  - export LD_LIBRARY_PATH=“$LD_LIBRARY_PATH:~/anaconda/lib”  —> work now.

 - 需要把 python search path 也設到 .bashrc (anaconda install 自動加入) or .tcshrc; 以及 LD_LIBRARY_PATH

 - 檢討是否還是用 equival 來 install 不同版本的 python.

-  重新整理 .bashrc, .bash_aliases, and .tcshrc —> add python search path and LD_LIBRARY_PATH

 —> 不過 LD_LIBRARY_PATH 的設定後反而後讓 make all failed!  所以只能在已 compile 之後才加上。

 

Set 2:  CPU only, with python link.  has problem!!

 

 

 

 

 

 

沒有留言:

張貼留言

追蹤者