notebook.community

Edit and run

Notes on linux in the year of 2018

Author: Yue-Wen FANG

Contact: fyuewen@gmail.com

Revision history: created in 11th, January, at Kyoto

1. About texlive in linux

After installing texlive into your linux system, you may still have problem when compling tex files (for example, when compling the manuscript for American Physica Society, I got this error: `revtex4-1.cls' not found. In avoid of this error, just install the texlive-publishers. Although it's a little bit large, but it's quite useful.

In Ubuntu, jsut type:

sudo apt-get install texlive-publishers

2. the solution of 'cannot change locale (UTF-8): No such file or directory'

Linux (what I mean here is Ubuntu) warning: setlocale: LC_CTYPE: cannot change locale (UTF-8): No such file or directory

sudo vi /etc/environment

add these lines...

LANG=en_US.utf-8 LC_ALL=en_US.utf-8





In [ ]:



    


## 2. All about ssh problems

2.1 Login the Ubuntun Linux built in the virtualbox with a windows 10 host

In my windows 10 computer, I have a ubuntun linux (version is 17.10, let's call this linux 'toutou') that was virtualized in the virtualbox. These systems are frequently used in my daily research. Hence, ssh is quite necessary for me.

This problem can be diveded into two problems: 1) login the ubuntu 'toutou' through the host windows system; 2) login the ubuntu 'toutou' from anywhere that is out of the windows system.

I show the methods that work for my instituation.

2.1.1 for the first problem

Firstly, set up 'VirtualBox Port Forwarding'. It is easy trough the VirtualBox->Settings-->Network --> Nat --> Advanced --> Port Forwarding --> set the Host Port as 3022, set the Guest Port as 22. When doing this, it's better to keep ubuntu system power off. After finishing the abve operations, login the ubuntu system.

Secondly, in the ubuntu 'toutou' terminal, use command 'ifconfig' (alternatively, we can use 'ip a' for analogous information), and you will get some interformation about the IP and mac address of the ubuntu system. Here, I show the my information,(considering safety, the details are removed/changed):





In [ ]:



    


enp0s3: flags=2743<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.1.20.15  netmask 255.255.255.0  broadcast 10.1.20.255
        inet6     prefixlen 64  scopeid 0x20<link>
        ether    txqueuelen 1000  (Ethernet)
        RX packets 132253  bytes 190923436349 (1.9 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 752342  bytes 1142345736 (114.8 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=93<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0 #127.0.0.1 is what we need
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 352118  bytes 184670313 (184.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 352118  bytes 184670313 (184.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Thirdly, go to the windows host, open a Git Bash or anything else like Cygwin, ssh user@127.0.0.1 (If there is no reponse, please check you have ssh server in the ubuntu; install it and restart the ssh server, try again)

2.1.2 for the second problem

I tried many solutions in the stackflow, but all failed. Accidently I solved it by myself just at the moment. Actually it's very simple.

Open the GitBash in my host windows, use commad 'ifconfig.exe' (or any other useful command that can return the network information). In my case, the information is





In [ ]:



    


$ ipconfig.exe

Windows IP Configuration


Ethernet adapter Ethernet:

   Connection-specific DNS Suffix  . : net
   IPv4 Address. . . . . . . . . . . : 10.202.71.201
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :  

Ethernet adapter VirtualBox Host-Only Network #2:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . :  
   IPv4 Address. . . . . . . . . . . : 192.168.0.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

The IP address '10.202.71.201' is what we need. Now, go to any other computers, for example, a Mac. Open the termianl in the Mac, and use command

ssh 3022 user@10.202.71.201

3. GLIBC errors: GLIBC_2.4 not found

I have encountered this problem for several times (particularlly when using import pymatgen). If the uer is also a admin of the computer or supercomputer, it is easy to solove by updating the GLIBC version. Otherwise, we need find a work-around. (In comet for pymatgen, I solved this GLIBC_2.4 problem by using 'pip install --user pymatgen')

I found a guy has provided a solution: see ref

Brifely,

$ export CFLAGS="-I. -include glibc_version_fix.h"

$ python setup.py build_ext





In [ ]:



    


(py27) ywfang@mac  ~/vaspwork/PCOO/conventional-40atom/7GPa/allin 08:19:06 >readelf -s /home/ywfang/miniconda3/lib/python3.6/site-packages/pymatgen/util/coord_cython.cpython-36m-x86_64-linux-gnu.so  | grep GLIBC_2.14
    88: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy@GLIBC_2.14 (4)
   801: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy@@GLIBC_2.14
(py27) ywfang@mac  ~/vaspwork/PCOO/conventional-40atom/7GPa/allin 08:19:26



    











In [ ]:



    


(py27) ywfang@mac  ~/vaspwork/PCOO/conventional-40atom/7GPa/allin 08:19:26 >readelf -V /home/ywfang/miniconda3/lib/python3.6/site-packages/pymatgen/util/coord_cython.cpython-36m-x86_64-linux-gnu.so

Version symbols section '.gnu.version' contains 170 entries:
 Addr: 0000000000001eb4  Offset: 0x001eb4  Link: 3 (.dynsym)
  000:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  004:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  008:   2 (GLIBC_2.2.5)   0 (*local*)       0 (*local*)       0 (*local*)
  00c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  010:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  014:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  018:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  01c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  020:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  024:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  028:   2 (GLIBC_2.2.5)   0 (*local*)       0 (*local*)       3 (GLIBC_2.4)
  02c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  030:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  034:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  038:   0 (*local*)       2 (GLIBC_2.2.5)   0 (*local*)       0 (*local*)
  03c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  040:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  044:   0 (*local*)       0 (*local*)       2 (GLIBC_2.2.5)   0 (*local*)
  048:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  04c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  050:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  054:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  058:   4 (GLIBC_2.14)    0 (*local*)       0 (*local*)       0 (*local*)
  05c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  060:   0 (*local*)       0 (*local*)       0 (*local*)       2 (GLIBC_2.2.5)
  064:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  068:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  06c:   0 (*local*)       0 (*local*)       5 (GLIBC_2.3.4)   0 (*local*)
  070:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  074:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  078:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  07c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  080:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  084:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  088:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  08c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  090:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  094:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  098:   2 (GLIBC_2.2.5)   0 (*local*)       0 (*local*)       0 (*local*)
  09c:   0 (*local*)       0 (*local*)       0 (*local*)       0 (*local*)
  0a0:   0 (*local*)       0 (*local*)       0 (*local*)       1 (*global*)
  0a4:   1 (*global*)      1 (*global*)      1 (*global*)      1 (*global*)
  0a8:   1 (*global*)      1 (*global*)

Version needs section '.gnu.version_r' contains 1 entries:
 Addr: 0x0000000000002008  Offset: 0x002008  Link: 4 (.dynstr)
  000000: Version: 1  File: libc.so.6  Cnt: 4
  0x0010:   Name: GLIBC_2.3.4  Flags: none  Version: 5
  0x0020:   Name: GLIBC_2.14  Flags: none  Version: 4
  0x0030:   Name: GLIBC_2.4  Flags: none  Version: 3
  0x0040:   Name: GLIBC_2.2.5  Flags: none  Version: 2

4. OpenBLAS blas_thread_init: pthread_create: Resource temporarily unavailable

Very recently, I got an error when 'import numpy' in Comet----OpenBLAS blas_thread_init: pthread_create: Resource temporarily unavailable

This error I got was a machine dependent problem becuase Comet prevent users running production work.

In Comet, there are two solutions:

  • vim .bashrc and set "export OMP_NUM_THREADS = 2" (2 can be changed to larger number dependending your computer)

  • Created an interaction session by using command "srun --partition=debug --pty --nodes=1 --ntasks-per-node=24 -t 00:30:00 --wait=0 --export=ALL /bin/bash", we can also start this session on Shared node by using "srun --partition=shared --pty --nodes=1 --ntasks-per-node=20 -t 00:30:00 --wait=0 --export=ALL /bin/bash"

It's better to use the former method because in the latter method, if we exit the terminal then the session also ends.

5. Stuck on the booting interface of Ubuntu 17.10

Yesterday, I found my Ubuntu 17.10 was stucked, although I can login the computer from other computers through ssh channel.

The solution was pointed out by stackexchange forum

For me, I logined the comptuer with Ubuntu 17.10 from my mac, and excuete:

sudo dpkg --configure -a

sudo apt-get upgrade

sudo reboot

6. Upgrading Ubuntu OS

Recently, Ubuntu 18.04.1 LTS was relased, to upgrade my 17.10 system, I excute 'do-release-upgrade'.

7. Install Julia in Ubuntu

  • Firstly, let us add the PPA (personal package archive) and updates the package index in the repository.

sudo apt-get add-repository ppa:staticfloat/juliareleases

sudo apt-get update

  • Secondly, install Julia

sudo apt-get install Julia

  • Finally, check the version of Julia

julia --version

For uninstalling Julia, just use 'sudo apt-get remove julia'

8. Upload large data to google drive from supercomputer or personal computers

I found a very powerful way to upload large data to google drive directly in Supercomputer servers （Linux）.

  1. Download the gdrive code:

wget https://docs.google.com/uc?id=0B3X9GlR6EmbnWksyTEtCM0VfaFE&export=download

  1. You should see a file in your home directory called something list uc=0B3X9GlR6EmbnWksyTEtCM0VfaFE. Rename this file to gdrive.

  2. chmod +x gdrive

  3. add the path of gdrive to your environment (.bashrc)

  4. upload file using the command: gdrive upload file

Ref: https://olivermarshall.net/how-to-upload-a-file-to-google-drive-from-the-command-line/





In [ ]:

Content source: yw-fang/readingnotes

Similar notebooks:

notebook.community | gallery | about