In [ ]:
conda create -n aiida-debug python=2.7 #set a veritual environment
In [ ]:
conda activate aiida-debug
#sometimes in mac, such a command might be requested
# sudo ln -s /Users/ywfang/miniconda3/etc/profile.d/conda.sh /etc/profile.d/conda.sh
In [ ]:
conda install postgresql
Now, we need download the aiida_core v0.11.0 and put it in the folder you like. For me, the directory of the aiida_core is '/Users/ywfang/FANG/study/software/all-aiida'
In [ ]:
git clone git@github.com:aiidateam/aiida_core.git
In [ ]:
cd aiida_core
In [ ]:
git checkout -b v0.12.1
In [ ]:
pip install .
#make sure that you have pip, if it is not available,
#use 'conda install pip'
In [ ]:
In [ ]:
Run 'verdi daemon start', the computer will remind you that you haven't set up the daemon user, then just follow it (As you will see below, I use my email address as the default user).
Please check the status of daemon:
aiida allows multiple profiles for a same user, but we need set one of them as a default. One profile corresponds to a data
In [ ]:
verdi profile setdefault verdi aiida_profile_201808
In [ ]:
verdi profile setdefault daemon aiida_profile_201808 #
In [ ]:
git clone git@github.com:DropD/aiida-vasp.git
After git, you'll see a new folder named aiida-vasp
In [ ]:
cd aiida-vasp/
In [ ]:
git checkout develop #switch to develop version
pymatgen parser setting: this step is quite important for aiida-vasp plugin. Although I don't like this feature very much because I usually forgot this step. Luckily, Abel have reminded me for many times ( at least 5, -_- ) in the latest month. Great thanks to him.
vi aiida_vasp/calcs/vasp.py, change the line
default_parser = 'vasp.vasp'
into
default_parser = 'vasp.pymatgen'
In [ ]:
pip install . # installed successfully
In [ ]:
In [ ]:
git clone git@github.com:abelcarreras/aiida-phonopy.git #this code was developped by Abel
You then see a folder "aiida-phonopy"
In [ ]:
cd aiida-phonopy
git checkout development #switch to develop version
In [ ]:
pip install . # installed successfully
Since in 'verdi quicksetup', I didn't use the passward, hence I run 'verdi setup aiida1', and reset my profile. The profile information is stored in a file like:
~/.aiida/config.json
For the supercomputer you want to perform calculations, before settupng up the computer, you have to prepar a password-less connection to the computer. (use ssh-keygen, there are many blogs on this topic.) Here, I skip this step, and go to the 'verdi comptuer setup' directly.
In [ ]:
(aiida) h44:aiida_plugins ywfang$ verdi computer setup
At any prompt, type ? to get some help.
---------------------------------------
=> Computer name: stern-lab
Creating new computer with name 'stern-lab'
=> Fully-qualified hostname: 88.88.88.88 #here is the IP address of your computer
=> Description: go to stern-lab from mac
=> Enabled: True
=> Transport type: ssh
=> Scheduler type: sge
=> shebang line at the beginning of the submission script: #!/bin/bash
=> AiiDA work directory: /home/ywfang/aiida_run_mac
=> mpirun command: mpirun -np {tot_num_mpiprocs}
=> Text to prepend to each command execution:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ------------------------------------------
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ------------------------------------------
export PATH=/usr/local/calc/openmpi-1.10.2-ifort-togo/bin/:$PATH #change these two lines according to your system
export LD_LIBRARY_PATH=/opt/intel/Compiler/11.1/069/lib/intel64/:$LD_LIBRARY_PATH
=> Text to append to each command execution:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ------------------------------------------
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ------------------------------------------
Computer 'stern-lab' successfully stored in DB.
pk: 1, uuid: f54386aa-0b7f-4576-8faa-666d9429980c
Note: before using it with AiiDA, configure it using the command
verdi computer configure stern-lab
(Note: machine_dependent transport parameters cannot be set via
the command-line interface at the moment)
Set up local computer as localhost
Appended note in March 5th 2018
Reference: https://github.com/ltalirz/aiida-zeopp
It is also possible to setup a local computer as a host to do computations. For example, I show the lcoal computer information:
Computer name: localhost
- PK: 4
- UUID: ..................
- Description: my local computer
- Hostname: localhost
- Enabled: True
- Transport type: local
- Scheduler type: direct
- Work directory: /home/ywfang/aiidalocal-run
- mpirun command:
- Default number of cpus per machine: 8
- Used by: 4 nodes
- prepend text: # No prepend text.
- append text: # No append text.
After setting up the local computer, remeber to configure this computer.
Here, correspondingly I give the code information in my local computer as an example:
- PK: 186209
- UUID:
- Label: phonopy
- Description: phonopy in localhost
- Default plugin: phonopy.phonopy
- Used by: 1 calculations
- Type: remote
- Remote machine: localhost
- Remote absolute path: /home/ywfang/miniconda3/envs/aiida/bin/phonopy
- prepend text: # No prepend text.
- append text: # No append text.
A comment to the 'direct' scheduler:
The direct scheduler, to be used mainly for debugging, is an implementation of a scheduler plugin that does not require a real scheduler installed, but instead directly executes a command, puts it in the background, and checks for its process ID (PID) to discover if the execution is completed.
Warning
The direct execution mode is very fragile. Currently, it spawns a separate Bash shell to execute a job and track each shell by process ID (PID). This poses following problems:
PID numeration is reset during reboots; PID numeration is different from machine to machine, thus direct execution is not possible in multi-machine clusters, redirecting each SSH login to a different node in round-robin fashion; there is no real queueing, hence, all calculation started will be run in parallel.
In [ ]:
(aiida) h44:aiida_plugins ywfang$ verdi computer configure stern-lab
Configuring computer 'stern-lab' for the AiiDA user 'fyuewen@gmail.com'
Computer stern-lab has transport of type ssh
Note: to leave a field unconfigured, leave it empty and press [Enter]
=> username = ywfang
=> port = 22
=> look_for_keys = ~/.ssh/id_rsa
Error in the inserted value: look_for_keys must be an boolean
=> look_for_keys =
=> key_filename = ~/.ssh/id_rsa
=> timeout = 60
=> allow_agent =
=> proxy_command =
=> compress = False
=> gss_auth = no
=> gss_kex = no
=> gss_deleg_creds = no
=> gss_host = 88.88.88.88
=> load_system_host_keys = True
=> key_policy = WarningPolicy
Configuration stored for your user on computer 'stern-lab'.
In [ ]:
(aiida) h44:aiida_plugins ywfang$ verdi computer test stern-lab
Testing computer 'stern-lab' for user fyuewen@gmail.com...
> Testing connection...
> Getting job list...
`-> OK, 30 jobs found in the queue.
> Creating a temporary file in the work directory...
>>>>>
>>>>>>.....
..........
.........
[Deleted successfully]
Test completed (all 3 tests succeeded)
verdi computer list
to get a list of existing computers, and:
verdi computer show COMPUTERNAME
to get detailed information on the specific computer named COMPUTERNAME. You have also the:
verdi computer rename OLDCOMPUTERNAME NEWCOMPUTERNAME
and:
verdi computer delete COMPUTERNAME
In [ ]:
Basically there are two kinds of code for aiida. One is local code, the other one is the socalled remote code that is in a supercomputer or cluster. Please ref to (aiida)[http://aiida-core.readthedocs.io/en/release_v0.11.0/get_started/index.html] about their differences.
Here I show a example to setup a remote code (vasp):
In [ ]:
(aiida) h44:aiida_plugins ywfang$ verdi code setup
At any prompt, type ? to get some help.
---------------------------------------
=> Label: vasp
=> Description: vasp541 at stern-lab
=> Local: False
=> Default input plugin: vasp.vasp # you can use verdi calculation plugins to check what plugins you have
=> Remote computer name: stern-lab
=> Remote absolute path: /usr/local/vasp/vasp541
=> Text to prepend to each command execution
FOR INSTANCE, MODULES TO BE LOADED FOR THIS CODE:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ------------------------------------------
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ------------------------------------------
=> Text to append to each command execution:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ------------------------------------------
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ------------------------------------------
Code 'vasp' successfully stored in DB.
pk: 1, uuid: 03875075-0cc1-4938-8943-c46d3ee2aecd
Attention For setting up code for phonopy and phono3py, the "Defult input plugins" are phonopy.phonopy and phonopy.phono3py respectively.
Some useful commands:
verdi code rename "ID" (for example, vasp@stern-lab is an ID)
verdi code list
verdi code show "ID"
verdi code delete "ID"
The example I tested comes from the aiida-vasp plugin (prabably I made some small modifications):
In [ ]:
import click
import numpy
@click.command()
@click.option('--paw-family', type=str, default='vasp-test')
@click.option('--import-from', type=click.Path(), default='.')
@click.option('--queue', type=str, default='')
@click.argument('code', type=str)
@click.argument('computer', type=str)
def test_vasp(paw_family, import_from, queue, code, computer):
load_dbenv_if_not_loaded()
from aiida.orm import CalculationFactory, Code
if import_from:
import_paws(import_from, paw_family)
# try:
paw_si = get_paws(paw_family)
# except ValueError as err:
# click.echo(err.msg, err=True)
# raise ValueError(
# 'give a valid family or import a new one (run with --help)')
vasp_calc = CalculationFactory('vasp.vasp')()
vasp_calc.use_structure(create_structure2())
vasp_calc.use_kpoints(create_kpoints())
vasp_calc.use_parameters(create_params())
code = Code.get_from_string('{}@{}'.format(code, computer))
vasp_calc.use_code(code)
# vasp_calc.use_paw(paw_in, 'In')
# vasp_calc.use_paw(paw_as, 'As')
vasp_calc.use_paw(paw_si, 'Si')
vasp_calc.set_computer(code.get_computer())
vasp_calc.set_queue_name(queue)
vasp_calc.set_resources({
'num_machines': 1,
'parallel_env': 'mpi*',
'tot_num_mpiprocs': 16
})
vasp_calc.label = 'Test VASP run'
vasp_calc.store_all()
vasp_calc.submit()
vasp_calc = CalculationFactory('vasp.vasp')
#from aiida_vasp.parsers.pymatgen_vasp import PymatgenParser
#vasp_calc.set_parser_cls(PymatgenParser)
#vasp_calc.use_settings(DataFactory('parameter')(dict={
# 'pymatgen_parser': {
# 'exception_on_bad_xml': False,
# 'parse_dos': False
# }
# }
#))
def load_dbenv_if_not_loaded():
from aiida import load_dbenv, is_dbenv_loaded
if not is_dbenv_loaded():
load_dbenv()
def get_data_cls(descriptor):
load_dbenv_if_not_loaded()
from aiida.orm import DataFactory
return DataFactory(descriptor)
def create_structure():
structure_cls = get_data_cls('structure')
structure = structure_cls(
cell=numpy.array([[0, .5, .5], [.5, 0, .5], [.5, .5, 0]]) * 6.058, )
structure.append_atom(position=(0, 0, 0), symbols='In')
structure.append_atom(position=(0.25, 0.25, 0.25), symbols='As')
return structure
def create_structure2():
import numpy as np
from aiida.orm import DataFactory
StructureData = DataFactory('structure')
a = 5.404
cell = [[a, 0, 0],
[0, a, 0],
[0, 0, a]]
symbols=['Si'] * 8
scaled_positions = [(0.875, 0.875, 0.875),
(0.875, 0.375, 0.375),
(0.375, 0.875, 0.375),
(0.375, 0.375, 0.875),
(0.125, 0.125, 0.125),
(0.125, 0.625, 0.625),
(0.625, 0.125, 0.625),
(0.625, 0.625, 0.125)]
structure = StructureData(cell=cell)
positions = np.dot(scaled_positions, cell)
for i, scaled_position in enumerate(scaled_positions):
structure.append_atom(position=np.dot(scaled_position, cell).tolist(),
symbols=symbols[i])
return structure
def create_kpoints():
kpoints_cls = get_data_cls('array.kpoints')
return kpoints_cls(kpoints_mesh=[8, 8, 8])
def create_params():
param_cls = get_data_cls('parameter')
return param_cls(dict={
'SYSTEM': 'InAs',
'EDIFF': 1e-5,
'ISMEAR': 0,
'SIGMA': 0.05,
'ENCUT': '280.00 eV',
'LEPSILON': '.TRUE.'
})
def import_paws(folder_path, family_name):
load_dbenv_if_not_loaded()
from aiida.orm import DataFactory
paw_cls = DataFactory('vasp.paw')
paw_cls.import_family(
folder_path, familyname=family_name, family_desc='Test family')
def get_paws(family_name):
load_dbenv_if_not_loaded()
from aiida.orm import DataFactory
paw_cls = DataFactory('vasp.paw')
#paw_in = paw_cls.load_paw(family=family_name, symbol='In')[0]
#paw_as = paw_cls.load_paw(family=family_name, symbol='As')[0]
paw_si = paw_cls.load_paw(family=family_name, symbol='Si')[0]
return paw_si
if __name__ == '__main__':
test_vasp()
The command to run this job:
"python run_vasp_old-boston.py --import-from /Users/ywfang/FANG/study/software/all-aiida/aiida_plugins/test-example-for-mac/VASP_TEST/Si vasp boston-lab"
Check the status of the calculations
In [ ]:
(aiida) h44:test-example-for-mac ywfang$ verdi calculation list -a
# Last daemon state_updater check: 0h:00m:07s ago (at 18:02:35 on 2018-01-19)
PK Creation State Sched. state Computer Type
---- ---------- -------- -------------- ---------- ---------
18 4m ago FINISHED DONE boston-lab vasp.vasp
Total results: 1
In [ ]:
In [ ]:
In [ ]:
In [ ]:
In [ ]:
In [ ]:
In [ ]:
In [ ]:
"""
(*Take KF compound as an example*)
(*In the unit cell of cubic KF,
there are 8 atoms, half of them are K, and the other half are F*)
This is the unit cell structure:
F K
1.0
5.3683051700000002 0.0000000000000000 0.0000000000000000
0.0000000000000000 5.3683051700000002 0.0000000000000000
0.0000000000000000 0.0000000000000000 5.3683051700000002
F K
4 4
Direct
0.0000000000000000 0.0000000000000000 0.0000000000000000
0.0000000000000000 0.5000000000000000 0.5000000000000000
0.5000000000000000 0.0000000000000000 0.5000000000000000
0.5000000000000000 0.5000000000000000 0.0000000000000000
0.5000000000000000 0.0000000000000000 0.0000000000000000
0.5000000000000000 0.5000000000000000 0.5000000000000000
0.0000000000000000 0.0000000000000000 0.5000000000000000
0.0000000000000000 0.5000000000000000 0.0000000000000000
"""
#(*using "phonopy --symmetry --tol=0.01" to get the primitive psocar*)
"""
The primitive poscar is
F K
1.0
0.0000000000000000 2.6841525850000001 2.6841525850000001
2.6841525850000001 0.0000000000000000 2.6841525850000001
2.6841525850000001 2.6841525850000001 0.0000000000000000
1 1
Direct
0.0000000000000000 0.0000000000000000 0.0000000000000000
0.5000000000000000 0.5000000000000000 0.5000000000000000
"""
In [14]:
import numpy as np
uc_vector = np.matrix([
[5.3683051700000002, 0, 0],
[0, 5.3683051700000002, 0],
[0, 0, 5.3683051700000002]
])
uc_vector
Out[14]:
In [15]:
primitive_vector = np.matrix([
[0, 2.6841525850000001, 2.6841525850000001],
[2.6841525850000001, 0, 2.6841525850000001],
[2.6841525850000001, 2.6841525850000001, 0]
])
primitive_vector
Out[15]:
In [16]:
x = np.linalg.solve(uc_vector.T, primitive_vector.T)
x
Out[16]:
The solved x is the primitive_aixs that can be used in the phononpy calculations
In [ ]:
#the conventional cell of B contains 36 atoms
B
1.0
4.8786175399999996 0.0000000000000000 0.0000000000000000
-2.4393087699999998 4.2250067299999996 0.0000000000000000
0.0000000000000000 0.0000000000000000 12.5104310900000009
B
36
Direct
0.8810421900000001 0.1189578100000000 0.1082863700000000
0.2379156300000000 0.1189578100000000 0.1082863700000000
0.8810421900000001 0.7620843700000000 0.1082863700000000
0.7856244800000000 0.5712489599999999 0.2250469600000000
0.7856244800000000 0.2143755200000000 0.2250469600000000
0.4287510400000000 0.2143755200000000 0.2250469600000000
0.4692953000000000 0.5307047000000000 0.3089958100000000
0.0614094100000000 0.5307047000000000 0.3089958100000000
0.4692953000000000 0.9385905900000000 0.3089958100000000
0.1973713700000000 0.3947427400000000 0.0243375300000000
0.1973713700000000 0.8026286300000001 0.0243375300000000
0.6052572600000000 0.8026286300000001 0.0243375300000000
0.5477088500000000 0.4522911500000000 0.4416197100000000
0.9045823000000000 0.4522911500000000 0.4416197100000000
0.5477088500000000 0.0954177000000000 0.4416197100000000
0.4522911500000000 0.9045823000000000 0.5583802900000000
0.4522911500000000 0.5477088500000000 0.5583802900000000
0.0954177000000000 0.5477088500000000 0.5583802900000000
0.1359619600000000 0.8640380400000001 0.6423291400000000
0.7280760700000000 0.8640380400000001 0.6423291400000000
0.1359619600000000 0.2719239300000000 0.6423291400000000
0.8640380400000001 0.7280760700000000 0.3576708600000000
0.8640380400000001 0.1359619600000000 0.3576708600000000
0.2719239300000000 0.1359619600000000 0.3576708600000000
0.2143755200000000 0.7856244800000000 0.7749530400000000
0.5712489599999999 0.7856244800000000 0.7749530400000000
0.2143755200000000 0.4287510400000000 0.7749530400000000
0.1189578100000000 0.2379156300000000 0.8917136300000000
0.1189578100000000 0.8810421900000001 0.8917136300000000
0.7620843700000000 0.8810421900000001 0.8917136300000000
0.8026286300000001 0.1973713700000000 0.9756624699999999
0.3947427400000000 0.1973713700000000 0.9756624699999999
0.8026286300000001 0.6052572600000000 0.9756624699999999
0.5307047000000000 0.0614094100000000 0.6910041900000000
0.5307047000000000 0.4692953000000000 0.6910041900000000
0.9385905900000000 0.4692953000000000 0.6910041900000000
#the primitive cell of B can be written as (use phonopy command to get this primitive cell)
B
1.0
2.4393087710850549 1.4083355756225715 4.1701436966666670
-2.4393087710850549 1.4083355756225715 4.1701436966666670
0.0000000000000000 -2.8166711512451430 4.1701436966666670
12
Direct
0.9893285600000000 0.3462020033333332 0.9893285600000000
0.3462020033333332 0.9893285600000000 0.9893285600000000
0.9893285600000000 0.9893285600000000 0.3462020033333332
0.0106714399999999 0.0106714399999998 0.6537979966666668
0.0106714399999999 0.6537979966666666 0.0106714400000000
0.6537979966666667 0.0106714400000000 0.0106714400000000
0.7782911033333333 0.3704052166666665 0.7782911033333333
0.3704052166666665 0.7782911033333333 0.7782911033333333
0.7782911033333333 0.7782911033333333 0.3704052166666666
0.2217088966666666 0.2217088966666666 0.6295947833333334
0.2217088966666666 0.6295947833333333 0.2217088966666668
0.6295947833333333 0.2217088966666665 0.2217088966666668
In [13]:
import numpy as np
uc_vector_B = np.matrix([
[4.8786175399999996, 0.0000000000000000, 0.0000000000000000],
[-2.4393087699999998, 4.2250067299999996, 0.0000000000000000],
[0.0000000000000000, 0.0000000000000000, 12.5104310900000009]
])
primitive_vector_B = np.matrix([
[2.4393087710850549, 1.4083355756225715, 4.1701436966666670],
[-2.4393087710850549, 1.4083355756225715, 4.1701436966666670],
[0.0000000000000000, -2.8166711512451430, 4.1701436966666670]
])
x_B = np.linalg.solve(uc_vector_B.T, primitive_vector_B.T)
x_B
#The structures of the two above examples are taken from Togo-sensei's phonon databse (I could have relaxed them with higher accuracy)
#the phonopy.conf in the database includes the information of primitive_aixs that can be referenced here as a comprision with my
#calculations here.
#for the definition of primitive_axis in phonopy, please see the manual or visit
#https://atztogo.github.io/phonopy/setting-tags.html#primitive-axis-or-primitive-axes
Out[13]:
In [ ]:
In [ ]:
In [ ]:
import urllib2 # needed for functions,classed for opening urls.
url = raw_input( "enter the url needed for downloading file(pdf,mp3,zip...etc)\n");
usock = urllib2.urlopen(url) #function for opening desired url
file_name = url.split('/')[-1] #Example : for given url "www.cs.berkeley.edu/~vazirani/algorithms/chap6.pdf" file_name will store "chap6.pdf"
f = open(file_name, 'wb') #opening file for write and that too in binary mode.
file_size = int(usock.info().getheaders("Content-Length")[0]) #getting size in bytes of file(pdf,mp3...)
print "Downloading: %s Bytes: %s" % (file_name, file_size)
downloaded = 0
block_size = 8192 #bytes to be downloaded in each loop till file pointer does not return eof
while True:
buff = usock.read(block_size)
if not buff: #file pointer reached the eof
break
downloaded = downloaded + len(buff)
f.write(buff)
download_status = r"%3.2f%%" % (downloaded * 100.00 / file_size) #Simple mathematics
download_status = download_status + (len(download_status)+1) * chr(8)
print download_status,"done"
f.close()
verdi calculation list to get the pk number corresponding to the raw data.
Run verdi shell
In [ ]:
#we can run do some interactive programming in the verdi shell
# verdi shell is a ipython based api of aiida
#Abel taught me about this analysis in Jan 19 2018
n = load_node(50442)
n.get_outputs_dict()
n.out.kappa
n.out.kappa.get_arraynames()
n.out.kappa.get_array('kappa')
array = n.out.kappa.get_array('kappa')
n.out.kappa.get_arraynames()
In [ ]: