This notebook forms part of a series on computational optical radiometry. The notebooks can be downloaded from Github. These notebooks are constantly revised and updated, please revisit from time to time.
The date of this document and module versions used in this document are given at the end of the file.
Feedback is appreciated: neliswillers at gmail dot com.
When asked which books she would recommend for newcomers to Python, my better half immediately said: find a real problem and good tools, then use Google search to solve the specific questions you run into. This made me think. Yes, Python is a computer programming language Yet, in its broader ecosystem is also so much more than just a language with syntax. Think of Python, its libraries and development environment as a set of problem solving tools, more than just a language. To use the tool set you need to know the language syntax, but you will find yourself quickly moving beyond the focus on a language towards focusing on solving the problem. It is almost as is the language will disappear as the focus shifts.
To be able to make the shift towards problem solving, you need a few building blocks in place:
Do get to know the language syntax, especially the built-in data structures such as lists, dictionaries, tuples and sets (also study the advanced specialized container data types in the collections module). Build a solid insight into common programming idioms, often referred to the 'the Pythonic way' (https://docs.python-guide.org/writing/style/) (this is very different from programming in traditional languages).
The really powerful tools in the Python toolset are the very many specialist packages (aka modules). Depending on your need these packages could be the scientific and data processing tools (numpy, scipy, sympy, pandas, etc.), web and database tools (django, flask, alchemy, etc.), machine learning (scikit-learn, PyTorch, Tensorflow), or specialist packages in astronomy, genomics, and many more fields. Most technical applications would benefit from a good grounding in numpy, scipy, sympy, and pandas, thereafter extending to other tools.
The 'best' development tool set can greatly aid in solving the problem. For scientific work the electronic lab book called Jupyter Lab is very popular. More conventional integrated development environment (IDE) include tools such as PyCharm (not free) and the free tools such as Spyder (available in Anaconda), or Visual Studio Code (do install the free Python extension by Microsoft). At a more basic level any text editor can be used, because Python uses standard ASCII files for scripting.
Python is most often installed as part of a bigger distribution that includes many more tools and packages than just the core Python interpreter. For scientific work the Anaconda distribution is the most popular --- being very complete, it is also very large with a more than a gigabyte footprint on the hard disk. The minimal installation which includes only the core Python functionality which is powerful but excludes the specialist packages.
Modern software development is almost unthinkable without a version control repository. The most popular tool here is git, with alternatives being subversion and mercurial. Do use a repository for development work, even if just locally on your computer. If you require off-site repository storage consider the likes of github or gitlab.
Take the trouble to learn the Markdown markup language, it is the de-facto standard in the code development world today. Scientific workers may also find benefit in using LaTeX for more advanced scientific and mathematical publication.
Python is most at home in a command window. Linux has several such terminals. Windows users can surely move beyond the standard cmd terminal to something like https://conemu.github.io/ or PowerShell.
Before discussing the Python installation procedure, consider potential development environments. These tools are critical to your success and positive experience with using Python.
Which integrated development environment (IDE) to install? You can write Python scripts with any ASCII editor (even notepad) and run the script in a command console. Some people find it more productive to work in a more powerful IDE environment. Finally, to build more comprehensive 'lab books' of your work with code, text, graphs, etc., use the IPython notebook.
There are a number of options listed here, starting from simple text editors, all the way to comprehensive IDE environments.
We will focus on the Windows OS here, because Linux users already know which tools they want to use. One note to Linux users: consider the Microsoft Visual Studio Code editor, it runs very well on Linux (it even has Vim keystroke bindings).
A lightweight way is to use a standard text editor (to write the script) and a command window (to execute the script). Good editors include:
https://code.visualstudio.com/ currently, this is the best general purpose editor (has nice extension for Python)
http://notepad-plus-plus.org/download/ free, general purpose editor.
https://atom.io/ free, general purpose editor.
http://www.sublimetext.com/ general purpose text editor, costs money.
https://wiki.python.org/moin/Vim if you know what vim is you don't need this. If your don't know about vim, don't bother.
There are several command windows to choose from. Windows users can use a third party command window (see here: http://aarontgrogg.com/blog/2015/07/31/a-better-windows-command-line-experience-comparing-powercmd-vs-console2-vs-consolez-vs-conemu-vs-cmder/). My current preference is conemu available here: https://conemu.github.io. Alternatively Windows users can use Microsoft's cmd.exe or Powershell. Linux users probably already have their favourite command window.
Specalised IDEs include (some might be included in large Python distributions):
http://pythonhosted.org/spyder/ good IDE already included in Anaconda.
https://www.jetbrains.com/pycharm/ free community version.
https://pytools.codeplex.com/ (Visual Studio plugin).
http://sourceforge.net/projects/pydev (Eclipse plugin).
The IPython notebook is a very powerful tool to record all your work and finally produce a report with all your documentation, code and results in one file. For more information on the IPython notebook see here:
http://nbviewer.ipython.org/github/NelisW/ComputationalRadiometry/blob/master/01-IPythonHintsAndTips.ipynb
Some Python installations are not kind to spaces in install folder names. Avoid spaces in any folder or filename to prevent issues.
Any Python path change only comes to effect when a new command window is opened. So if you make a change to the *.pth file, close the currently open Python session or Jupyter notebook, then close the command window, and finally open a new command window and run a new Python session.
The Python ecosystem comprises
You have two main options:
I would recommend the second option. There are a number of fairly complete distributions, including:
https://store.continuum.io/cshop/anaconda/ (my preferred distribution),
https://www.enthought.com/products/epd/,
https://code.google.com/p/pythonxy/,
http://www.sagemath.org/ (not very well known as a Python distribution).
Which to install: Python 2.7.X or Python 3.X? 2.7.X will be phased out some time in future (2020). 3.X is the later version with significant improvements over 2.7.X. If you have no compelling reason to use 2.7 (i.e., compatibility with some project), use 3.X.
Which to install: 32-bit or 64-bit?
If you want to remain compatible with other 32-bit installation packages or linking in with other 32-bit binary code, you must install the 32-bit version. If these restrictions do not apply, install the 64-bit version on a 64-bit PC.
**pyradi is only tested with Python 3.7**
https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/
Install Anaconda for Python 3.X.
http://conda.pydata.org/docs/_downloads/conda-cheatsheet.pdf
Some users prefer to have the Anaconda/Python paths permanently in the environment. This enables you to open Python from any command window, without using the Anaconda menus settings (When using the Anaconda menu, the paths are set for the one command window instance only).
Some installations had difficulty finding the required version of libiomp5md.dll. It appears that one installation had two (different?) version of the file:
where libiomp5md.dll
C:\ProgramData\Continuum\anaconda3\Library\bin\libiomp5md.dll
C:\Program Files (x86)\Common Files\Intel\Shared Files\cpp\Bin\Intel64\\libiomp5md.dll
Make sure that the Anaconda path comes first, above the other instances. Note that this may break the other application using the other instance.
Please read this excellent overview of conda environments:
https://www.freecodecamp.org/news/why-you-need-python-environments-and-how-to-manage-them-with-conda-85f155f4353c/
If you need to run different versions of Python (e.g., 2.7 and 3.6), consider using the conda environment infrastructure. A Conda environment is a new Python installation that lives alongside any other base installations or environment installations you already have. Essentially every Conda environment is a separated Python installation. When you want to use the specific python in the environment, you activate the environment. Activation prepares a command window and sets up the paths to the Python version in that specific environment.
http://conda.pydata.org/docs/using/envs.html
The underlying principle here is that you create a thin root installation environment that has the bare essentials to support conda and not much more. You then create conda environments where you can install full anaconda versions of the the appropriate Python revisions. You do the real work in the environments by activating the environment before running the python and packages in that environment. When you create environments in conda, it creates a more or less complete Python installation in the ..../envs/ directory in an existing miniconda or anaconda installation. So the multiple versions are installed side by side.
Activating an environment must be done in a command window; where the process of activation simply modifies the system level paths from the root environment to the newly activated environment. The new environment is active in the present command window until the environment is deactivated, which means that the original paths to the Python folders are restored back to the root environment.
It is recommended practice to keep a small root environment, solely as a tool to create new environments - all the real work is done in the specially created environments. This is a means to control which package versions for development, testing and deployment.
Install miniconda (Python version 3.7). Note that the choice of which Miniconda is installed only affects the root environment. Regardless of which version of Miniconda you install, you can still install both Python 2.7.x and other Python 3.x environments.
On Windows
it seems that win32api is used during a full anaconda install, so you might have to install
conda install pywin32it seems that installing conda requires admin rights for some of the tasks during the installation. The installation does complete, but there are warnings that for some tasks admin rights are required. Within the scope of my Python work, there never was a problem, but to install without any warning, start the command console as admin before installing conda/anaconda.
Then create environments for the versions you want to install (use whatever names you want, like py27, py36, pymc, myenv):
conda create --name py27 python=2.7 anaconda
conda create --name py36 python=3.6 anaconda
To activate the environment in a command window
activate py27
To deactivate the environment in a command window
decativate
To list all environments
conda info --envs
conda env list
To list modules in current environment
conda list -e
To search to see if a package is available in the anaconda repository (not your disk)
conda search numpy
Update an environment by activating it first and then updating - but note that there are two methods:
conda update anaconda - this command updates the anaconda meta-package as a list of packages tested against each other, known to work together.
conda update --all - this command updates individual packages to the individual latest versions, irrespective of compatibility status. It is likely that these packages will work together, but it is not tested and verified.
activate py36
conda update anacondaAfter being used for a while, conda can accumulate a lot of disk space, because it doesn’t remove old unused packages. To remove unused packages (-p), index caches (-i) and tarballs (-t), run
conda clean -pit
To upgrade/update to a specfic new version of anaconda in one of the environments, run
deactivate
conda update conda
activate py27
conda install anaconda=2.4.1
Conda loads the new environments in the *\Miniconda*\envs folder, inside the base environment (unless you instructed it to install elsewhere).
To remove a conda environment use:
conda remove --name py27 --all
conda info --envsSometimes it is necessary to recreate a new environment with exactly the same versions as an existing environment. The packages and versions can be captured in the requirement.txt file for the environment. This can be done in conda using the following command:
conda list --export > requirements.txt
When you are ready to distribute your package to other users, they can easily duplicate your environment and the associated dependencies with
conda create --name <envname> --file requirements.txt
https://github.com/rasbt/watermark is a jupyter extension that provides an extension with magics for printing date and time stamps, version numbers, and hardware information. Install the extension with
!pip install watermark
!conda install -c conda-forge watermark
and use the following to print the versions in a cell:
%load_ext watermark
%watermark -v -m -p numpy,scipy,pyradi,pandas -g -r -u -d -b -i
In [7]:
%watermark -v -m -p numpy,scipy,pyradi,pandas -g -r -u -d -b -i
Download the installer. In your terminal window type one of the below and follow the instructions:
Python 3.x:
bash Anaconda3-4.0.0-Linux-x86_64.sh
NOTE: Include the bash command even if you are not using the bash shell
https://conda-forge.org/ contains user-supplied packages not considered as part of the central anaconda distribution. A package in conda-forge can be installed with one of the following commands, which tells conda to look in the conda-forge repository:
conda install --channel https://conda.anaconda.org/conda-forge packagename
conda install -c conda-forge packagename
where packagename is the name of the package to be installed.
The packages available on conda-forge are shown here: http://conda-forge.org/feedstocks/
You can add the conda-forge channel to the list of available channels searched by conda when installing:
conda config --add channels conda-forgeThe most commonly-used packages are installed in the base anaconda install. You may want to later install packages that are not part of the base installer, but are available in the anaconda library.
This is done with the command (in this example case to install mayavi):
conda update conda
conda install mayavi
The new package install will automatically also install all depencency packages. Normally this is acceptable, but in some cases, the new package requires an earlier version of an already installed package - in which case the installed dependency package will downgrade the installed package to an older version. This may cause problems if another application requires the later version.
You can 'pin' package numbers to some release number to prevent automatic install changes to the release number, see here. If a package revision number is pinned, it will stay at the pinned revision, irrespective of all upgrade or dependency attempts to change it.
The --no-deps flag prevent automatic changes/upgrade/downgrade of intalled package dependencies. Note that this option will prevent and and all dependencies from downloading, so the newly installed package may not work correctly.
Conda attempts to install the newest versions of the requested packages. To accomplish this, it may update some packages that are already installed, or install additional packages. To prevent existing packages from updating, use the --no-update-deps option. This may force conda to install older versions of the requested packages, and it does not prevent additional dependency packages from being installed (http://conda.pydata.org/docs/commands/conda-update.html)
To force conda to install a specific version of a package, use the form conda install scipy=0.17.0, where the required version number is given after the equal sign.
If you wish to skip dependency checking altogether, use the --force option. This may result in an environment with incompatible packages, so this option must be used with great caution.
The conda update command also have these two dependency flags: --update-dependencies, --update-deps: Update dependencies (default: True). --no-update-dependencies, --no-update-deps Don't update dependencies (default: False).
Create a .condarc configuration file in your user home directory (C:\Users\[yourusername]), follow the instructions in the websites below. Place the following text in the .condarc file, but replace with the information relevant to your situation:
proxy_servers:
http: http://[yourusername]:[yourpassword]@[your proxy server]:[proxy port]
https: https://[yourusername]:[yourpassword]@[your proxy server]:[proxy port]
The difficult part is not creating the file, but finding out what the details for your situation are.
http://conda.pydata.org/docs/config.html
https://groups.google.com/a/continuum.io/forum/#!msg/anaconda/WQBn1c-spQ4/HVa1v83zAlEJ
Please note that this `.condarc` file is open text and the password is readable with a normal text editor. Change the actual password to a false password immediately after use!
Some of the notebooks use the Jupyter widgets. The widget manager must be installed manually after the anaconda install
jupyter labextension install @jupyter-widgets/jupyterlab-manager
https://www.npmjs.com/package/@jupyter-widgets/jupyterlab-manager
https://jupyterlab.readthedocs.io/en/stable/user/extensions.html
Jupyter supports multiple environments by running different kernels for each version. So you can run conda environments for Python 2.7 and Python 3.6 notebooks from the same server, just by selecting the kernel.
It seems that jupyter (ipython notebook) only runs from the root and not from an environment. So we follow a procedure here of installing jupyter in the root installation and then create different kernels for each of the different Python versions installed in the environments.
The idea here is to install multiple ipython kernels and here. Here are instructions if you are using anaconda:
install miniconda or anaconda. If you want only a basic Python capability in the root installation, install miniconda. On the other hand if you want a full anaconda capability in the root environment, install anaconda. Note that you will be installing full anaconda versions in the py27 and py35 environments. The only reason not to use miniconda is to have the full anaconda suite available at any moment, without activating a conda environment.
install jupyer in the miniconda root
deactivate
conda update conda
conda install --upgrade jupyterSuppose we want a new conda environment (named py36) with python 3.6 and set up Jupyter to work with py36.
# you might already have done the installations, don't repeat:
conda create -n py36 python=3.6 anaconda
conda install --upgrade <<any packages you need in this environment>>
# now continue with ipykernel installation in py36
activate py36
conda update anaconda
conda install notebook ipykernel
ipython kernel install
deactivateRepeat the the last procedure for all environments you want to use in Jupyter.
Note
Early in 2020 the above command ipython kernel install did not have the desired effect. In the end the following command did the job (mort is the environment name in this case):
python -m ipykernel install --user --name mort --display-name "Python (mort)"
The --name and --display-name options to ipython kernel install is the name of the kernel and the display name in the notebook. See ipython kernel install --help for more information or
here.
To see which kernels are available open jupyter and start a new notebook. Alternatively use the command:
jupyter kernelspec list
which will display the list of available kernels similar to the following:
Available kernels:
mort C:\Users\NWillers\AppData\Roaming\jupyter\kernels\mort
python3 C:\ProgramData\Anaconda3\share\jupyter\kernels\python3
These lines are paths to where the kernel is defined. Note that the two kernels are not defined in the same folder. In this case mort was created by the above user command. The Python3 kernel was created by the installer.
To remove a kernel use the command
jupyter kernelspec uninstall <kernel name>
where <kernel name> would be something like mort for the above list.
Don't remove the kernel installed by the installer.
When set up this way you can also use any environment by simply activating the respective version.
On occasion you may find that, when loading a notebook, you get a kernel error
The system cannot find the file specified
It seems to be created after environments are removed. I tried to run the following (as given in previous section), but it had no effect:
conda install notebook ipykernel
ipython kernel install
After google searches I found this solution, which worked:
python -m ipykernel install --user
this seems to (re?)install the kernel for the current user, using python instead of the ipython command show in the previous section.
https://github.com/jupyter/notebook/issues/2301 (nice detail here)
https://github.com/jupyter/notebook/issues/4079 (involves changing a json file, but the above worked, so I did not try this)
Python looks for installed packages in the \Lib\site-packages folder. If the package is not found in one of the many \Lib\site-packages folders, Python will fail importing the package.
It is possible to keep a copy of your package elsewhere, outside of any \Lib\site-packages folder, but then you have to tell Python about the package name and where the package is installed on your PC. To do this, we need to find the \Lib\site-packages folder and then place a *.pth file in this folder.
\Lib\site-packages foldersThe python distribution e.g., Anaconda, can be installed in a number of different locations. To determine where the conda environments on your PC is located, open a command window and type the following:
conda env list
Now we have to determine the location of the site-packages folder in the environments, from which Python can locate the available modules. This location could be one of the following (or similar):
C:\Users\YourUserName\Anaconda3\Lib\site-packages
C:\Miniconda3\Lib\site-packages
C:\ProgramData\Anaconda3\Lib\site-packages
In the conda environments the packages are installed in the ....\envs folder, inside the base environment, at locations with the following name structure:
`<where installed>\.\envs\<enironment name>\.\Lib\site-packages`
i.e., something like
C:\Users\YourUserName\Anaconda3\envs\py36VTK\Lib\site-packages
C:\Users\YourUserName\Anaconda3\envs\landlab\Lib\site-packages
C:\ProgramData\Anaconda3\envs\pymc\Lib\site-packages
*pth fileTo tell Python about your special installation folder, place a <module name>.pth file in the site-packages directory in the environment where this package must be available.
The name of the file must correspond to the name of the module <module name> and the file extension must be .pth. The file must contain the absolute path to the module on the first line. BTW the pth extension is short for path, which is what is in the file.
For example, if I cloned the https://github.com/NelisW/pyradi module from github to my hard disk at V:\work\WorkN\pyradi, I can place a file in the root and or environment site-packages folder(s) with the name pyradi.pth that contains the line V:\work\WorkN\pyradi\pyradi. Note that in this case pyradi is installed at a particular location, but the path written to the file is further down the folder structure with an additional pyradi. This is because the module is actually one level deeper.
If you use the .pth method, Python does not know if it is a local folder structure, a repository, or a pypi installation, just so long as it has legal Python module code, all is fine.
I have not done this yet, but in principle you can have many different versions of a module in different directories on your PC (e.g. for testing with different versions of Python, or with different functionality for whatever reason). You can then create different conda environments where each environment has its own unique and different .pth file, pointing to the specific code you want to use with that specific environment. The different .pth files must then be placed in each environment's own site-packages directory, to be used with that specific environment.
References:
https://docs.python.org/2/library/site.html
http://bob.ippoli.to/archives/2005/02/06/using-pth-files-for-python-development/
https://grahamwideman.wikispaces.com/Python-+site-package+dirs+and+.pth+files
http://stackoverflow.com/questions/15208615/using-pth-files
You only need to do this if you want to use pyradi for computational radiometry.
pyradi is no longer supported on PyPi.
You should really consider using a software version control process for your coding and writing work. There are several alternatives, but I prefer to use the current mainstream tool Git. So install git, use it to manage pyradi on your PC, but more importantly, use it to manage your own work afterwards.
If you are not already using git consider the following resources:
http://lmgtfy.com/?q=starting+git
https://git-scm.com/book/en/v1/Getting-Started
https://rogerdudler.github.io/git-guide/
http://sirupsen.com/starting-with-git/
https://product.hubspot.com/blog/git-and-github-tutorial-for-beginners
Search for git videos on YouTube.
There are also good books on Git.
If you use Linux, you probably already have git on the system and might also know how to use it.
Working on Windows I would recommend installing git from https://gitforwindows.org/, and then installing the Tortoise Git Explorer shell interface from https://tortoisegit.org/download/. Tortoise creates context menus on your Windows Explorer which makes it very easy to use for most git tasks.
To checkout the pyradi repo from github you do not need a github account. You only need to create a github account if you want to create repositories on GitHub. So initially, no need to get an account.
Go to https://github.com/NelisW/pyradi and look for the green button on the right side, near the top of the page and click on it. Then click on the little clipboard icon to load the link from the textbox to your local PC clipboard.
Open a Windows Explorer and create a new folder (e.g., your-master-folder) where you want to clone the pyradi repo (the folder can be anywhere you want, just be careful to not have spaces in the pathname). Right-click in the empty folder and then in the context menu, click on git clone.... This should open up a dialog box that already shows the pyradi URL pasted from the clipboard. The directory where the repo will be cloned is also shown. Don't change anything and click on the OK button. If all goes well a new window appear that shows the progress with cloning (downloading) the repo.
Also clone the pyradi data from https://github.com/NelisW/pyradi-data and (optionally) the pyradi docs from https://github.com/NelisW/pyradi-docs, using the same procedure as above. Clone in the same folder where pyradi is installed. Use exactly the same names for pyradi and pyradi-data. So when you are done there should be two or three folders depending on where you installed docs.
your-master-folder
pyradi
pyradi-data
pyradi-docs
At this point you have pyradi on your PC but your Python installation does not know about it yet. Continue with the procedure below to tell Python about your pyradi files' location in the pyradi.pth file.
If you want to update the locally cloned version later (and we recommend that you do this from time to time), right-click on the folder where pyradi is installed and click on the TortoiseGit line and then the Pull line. A dialog box should appear where you click on OK. This way you will always have the latest version of pyradi.
This option is second best because to get later versions you have to download the zip file(s) again and again. If you use Git, you can just 'pull'.
Go to https://github.com/NelisW/pyradi and look for the green button on the right side, near the top of the page; click on the green button. At the bottom of the small popup window, click on the Download Zip link to download the zip file to your PC.
Also download the pyradi data from https://github.com/NelisW/pyradi-data and (optionally) the pyradi docs from https://github.com/NelisW/pyradi-docs, using the same zip file download procedure as above.
Open a Windows Explorer and create a new folder (e.g., your-master-folder) where you want to unzip the pyradi repo (the folder can be anywhere you want, just be careful to not have spaces in the pathname). Unzip all two or three zip files into one this folder. So when you are done there should be two or three subfolders. Keep the three subfolder names exactly as shown, don't change them.
your-master-folder
pyradi
pyradi-data
pyradi-docs
At this point you have pyradi on your PC but your Python installation does not know about it yet. Continue with the procedure below to tell Python about your pyradi files' location in the python.pth file.
If you later want to instal updated versions, download the zip files and copy over the current files in the same directories.
Follow the procedure in the previous section to create a pyradi.pth file. The file must be named pyradi.pth and the first (and only line) in the file must be the path to where the pyradi module is available. On my PC this is V:\work\WorkN\pyradi\pyradi - note the second pyradi in the link to point further down the repository tree.
You do not have to tell Python about the documentation or data.
Now if you now start up a Python in a new command window, Python will locate the pyradi module by looking up the location on your PC from the pyradi.pth file.
pyradi should work without any additional package installations beyond Anaconda, except for the optional use of Mayavi. Mayavi is only required for the three-dimensional spherical plots, you may ignore the requirement otherwise. Most sources advise users to install Mayavi as part of a Python distribution, such as the Enthought Tool Set or Anaconda distributions. This is certainly an option for newcomers, but these distributions are not always current with the latest code package versions. Of the two, Anaconda appears to be quicker with the updates. Look for Anaconda (under the supervision of Travis Oliphant, one of the driving forces behind Numpy) or Enthought Tool Set, both of which contains contains Mayavi and other tools. You can find Mayavi (and other) installation packages on this site.
Use PyQt5 if you can. The pyradi library has been updated to work with PyQt5.
Anaconda 4.2 and later comes with PyQt5 only (PyQt4 on older versions). It is not possible to install PyQt4 and PyQt5 in the same environment. If you need PyQt4 under later versions of Anaconda you must uninstall PyQt5 and then reinstall PyQt4.
When uninstalling PyQt5, a number of other modules that rely on PyQt5 is also uninstalled, so you must consider the implications. You will loose these modules:
anaconda: 4.3.0-np111py27_0
anaconda-navigator: 1.4.3-py27_0
jupyter: 1.0.0-py27_3
matplotlib: 2.0.0-np111py27_0
pyqt: 5.6.0-py27_2
qt: 5.6.2-vc9_3 [vc9]
qtconsole: 4.2.1-py27_2
scikit-image: 0.12.3-np111py27_1
seaborn: 0.7.1-py27_0
spyder: 3.1.2-py27_0
First create a new environment and activate it:
conda create --name py27qt4 python=2.7 anaconda
activate py27qt4
Then uninstall qt:
(py27qt4)> conda uninstall qt
Then install PyQt4 from here:
conda install --channel https://conda.anaconda.org/conda-forge pyqt=4
which installs these modules:
libiconv: 1.14-vc9_4 conda-forge [vc9]
libxml2: 2.9.3-vc9_9 conda-forge [vc9]
pyqt: 4.11.4-py27_2 conda-forge
qt: 4.8.7-vc9_6 conda-forge [vc9]
vc: 9-0 conda-forge
I am not sure if you can reinstall Matplotlib now with PyQt. In my use case it was not necessary.
The PYTHONPATH environmental variable tells python where to look for modules when importing, it does not define the path to a *.py script to be executed. If you have a module in a directory outside the standard Python install directory (e.g., an experimental module in your HOME directory), add the path to the module to the PYTHONPATH environmental variable. The general advice is to not change the environmental variable permanently on operating system level, because of potential conflicts with other installed packages.
From inside Python execute the following command to add a new path to your current PYTHONPATH, which will not add the path permanently:
import sys
sys.path.append('/path/to/your/python/module')
To insert your new path at the front of PYTHONPATH (useful if you want to look in your local directories first, overriding existing installs):
sys.path.insert(0,'/path/to/your/python/module')
To get a list of the paths currently in PYTHONPATH:
import sys
print(sys.path)
If you are using/developing a module that is not located in .../site-packages/, but you want a path to the module on a more permanent basis in your PYTHONPATH, use a path configuration file. Path configuration files are usually placed in the .../site-packages/ directory (or a few other places) and can have any name, but with an extension of .pth.
Each line must contain a single path that will be appended to sys.path. New paths are appended to sys.path, modules in the added directories will not override standard modules.
It is not clear from the documentation, but the paths to the modules listed in the pht file must be relative to one of the site packages directories. These directories are where Python would look for modules. Absolute paths do not seem to work (at least on Windows, the are Linux examples that do have absolute paths). If your module is located on the same disk as .../site-packages/, then it appears necessary to provide a path relative to this .../site-packages/, i.e., something like ../../../pathtomodule. If the module is not on the same disk/drive as .../site-packages/, you should probably set up a secondary site-packages directory on the other disk (using addsitedir).
https://docs.python.org/2/library/site.html
https://docs.python.org/2/library/site.html#site.USER_SITE
https://docs.python.org/2/install/
https://stackoverflow.com/questions/15208615/using-pth-files
http://bob.ippoli.to/archives/2005/02/06/using-pth-files-for-python-development/
https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/RO_h0wD7sKE
https://stackoverflow.com/questions/10693706/creating-a-secondary-site-packages-directory-and-loading-packages-from-pth-fil
https://stackoverflow.com/questions/19914527/using-python-pth-files-assuming-no-access-to-site-packages-directory
https://docs.python.org/2/library/site.html#site.addsitedir
Use any of the above methods
https://stackoverflow.com/questions/17386880/does-anaconda-create-a-separate-pythonpath-variable-for-each-new-environment
Canopy users please see this
https://support.enthought.com/entries/23665767-How-do-I-set-PYTHONPATH-and-other-environment-variables-for-Canopy-
https://stackoverflow.com/questions/19917492/how-to-use-pythonpath
https://stackoverflow.com/questions/4580101/python-add-pythonpath-during-command-line-module-run
http://www.stereoplex.com/blog/understanding-imports-and-pythonpath
https://stackoverflow.com/questions/1489599/how-do-i-find-out-my-python-path-using-python
http://users-cs.au.dk/chili/PBI/pythonpath.html
http://docs.continuum.io/anaconda/ide_integration.html
In [8]:
import sys
print(sys.path)
To execute a python script from anywhere on your system, use the PATH environmental variable.
On Linux:
Add your script path to the system's PATH environmental variable:
export PATH=$PATH:/home/user/path/to/python/script.py
Add the following line as the first line (shebang line) of the script to tell Linux which interpreter to use:
#!/usr/bin/env python
Set up the script as an executable:
chmod +x /home/user/path/to/python/script.py
https://stackoverflow.com/questions/19917492/how-to-use-pythonpath
On Linux:
If the file is created in Windows, the line ending is not Linux friendly. Windows uses carriage return and line feed \r\n, whereas Linux uses just a line feed \n. This means that if the file is created on Windows and executed on Linux the operating system is searching to execute the file python\r, which is very different from the intended python. I tried converting the file from Windows to Linux format by using the fromdos command, but that did not fix the line ending problem. Opening and closing the file in nano did not help either. The only remedy was to (1) open the file in Linux (e.g. in nano), move to the top of the file, (2) manually type in the whole first line followed by Enter a few times, making very sure that a new Linux line ending is introduced, and (3) then remove the old line with the bad line ending.
On Windows:
Add the path to your script to your user or system PATH environmental variable. The Rapid Environment Editor tool makes it very easy. Once the path is set up, you can just double click on the script or run it in a command window.
cookiecutter seems to lead the pack:
https://cookiecutter.readthedocs.io/en/latest/
A command-line utility that creates projects from cookiecutters (project templates), e.g. creating a Python package project from a Python package project template.
conda install -c conda-forge cookiecutter
https://cookiecutter.readthedocs.io/en/latest/first_steps.html
https://github.com/mikeckennedy/cookiecutter-course
https://medium.com/worldsensing-techblog/project-templates-and-cookiecutter-6d8f99a06374
Some templates:
https://cookiecutter.readthedocs.io/en/latest/readme.html#a-pantry-full-of-cookiecutters
http://cookiecutter-templates.sebastianruml.name/
https://nsls-ii.github.io/scientific-python-cookiecutter/ -- nice overview
https://drivendata.github.io/cookiecutter-data-science/
https://github.com/mandeep/cookiecutter-pyqt5
https://github.com/selimb/cookiecutter-latex-article
https://docs.microsoft.com/en-us/visualstudio/python/using-python-cookiecutter-templates https://docs.microsoft.com/en-us/visualstudio/python/quickstart-04-python-in-visual-studio-project-from-cookiecutter
In [9]:
# to get software versions
# https://github.com/rasbt/watermark
# An IPython magic extension for printing date and time stamps, version numbers, and hardware information.
# you only need to do this once
# !pip install watermark
# conda install -c conda-forge watermark
#%load_ext watermark
%watermark -v -m -p numpy,scipy,pyradi,pandas -g -r -u -d -b -i
In [ ]: