Python Dev Env¶

SublimeText `virtualenv` Build System¶

NOTE: THIS IS FROM A FEW YEARS AGO, HAVEN'T USED SUBLIME VERY MUCH SINCE

TESTED & WORKED:

FILE:¶

File Name & Location:¶

ST2:
bash ~/Library/Application Support/Sublime Text 2/Packages/User/venv.sublime-build
ST3:
bash ~/Library/Application Support/Sublime Text 3/Packages/User/venv.sublime-build

File Contents:¶

SUBLIME TEXT 2:

USING PROJECTS:
(I.E. REQUIRE A PROJECT FILE TO BE SAVED)
bash { "cmd": ["$project_path/bin/python", "-u", "$file"] }
NOT USING PROJECTS:
bash { "cmd": ["$file_path/bin/python", "-u", "$file"] }

SUBLIME TEXT 3:

USING PROJECTS:
(I.E. REQUIRE A PROJECT FILE TO BE SAVED)
bash { "shell_cmd": "$project_path/bin/python -u \"$file\"" }
NOT USING PROJECTS:
bash { "shell_cmd": "$file_path/bin/python -u \"$file\"" }

Python Setup :: Detail of how it works¶

1 - pyenv¶

The setup works as follows:

python command is symlinked to python3 so that python3 is the global python
two environments, jupyter3 and tools3 are set up using pyenv-virtualenv
- NOT virtualenvwrapper! it doesn’t find these two virtualenvs!
- to activate them run: pyenv activate jupyter3
- to deactivate run: pyenv deactivate
jupyter3 & tools3 are added to the ‘global’ pythons, so the packages installed in these virtualenvs are available systemwide
the idea is for any “projects” to be set up using virtualenvwrapper
- the virtualenvs so created will be activated using workon xxx
- NOTE: difference between pyenv activate (i.e. using pyenv-virtualenv) and mkvirtualenv (i.e. using virtualenvwrapper):
- pyenv activate
  - these are 'global level' environments of pyenv itself
  - e.g. the two main environments jupyter3 and tool3 created as per above and accessible from everywhere
  - the idea is to install these 'global' pyenv environments for main tools to use across the whole system
- mkvirtualenv
  - these are 'project level' environments
  - created using mkvirtualenv (i.e. virtualenvwrapper)
  - ideally created in the venvs folder (the folder declared in PROJECT_HOME declared in .bashrc or in my case, .venvsconf in folder dotfiles)
for some reason the syspip(){} doesn’t work, so to install anything in the global environment, the PIP_REQUIRE_VIRTUALENV environment variable has to be reset

some virtualenvwrapper commands:
REF: VirtualEnvWrapper Docs Page

```
mkvirtualenv env1
lsvirtualenv
lssitepackages
workon env1
rmvirtualenv
cpvirtualenv
deactivate
cdvirtualenv
```

Examples:¶

Say I want to start a new project which I'll call proj3 with Python3.
- Running mkproject proj3 will create :
  - a virtualenv with Python3 (default) at ~/.ve/proj3
  - and an empty project directory at ~/venvs/proj3
Now say I just opened the terminal and want to work on my new proj3.
- Running workon proj3 will:
  - activate the virtualenv ~/.ve/proj3
  - and change the current directory to ~/workspace/proj3.

2 - Docker¶

Refer to this note: Python - Ananconda in Docker Container

Python Setup - Step by step¶

Python Setup 2018

(Installation Process used 2017-11-01 & again on 2018-12-18 on a fresh installation of macOS)

First install pyenv and its extensions to manage virtual environments:

brew install pyenv
brew install pyenv-virtualenv
brew install pyenv-virtualenvwrapper

virtualenvs
i. create directory for the virtualenvs :
mkdir ~/.ve
ii. create directory for the projects :
mkdir ~/venvs
Add the following to the bashrc
Note the last line being commented out
(In my setup these are declared in the file .venvsconf and sourced through .bashrc)
```
export WORKON_HOME=~/.ve
export PROJECT_HOME=~/venvs
eval "$(pyenv init -)"
# pyenv virtualenvwrapper_lazy
```
Restart the terminal or source .bashrc
Make sure that the following tools are installed (with homebrew):
```
readline
openssl
xz
```
- Also, on macOS Mojave or higher might need to install the additional SDK headers:
  Common build problems · pyenv/pyenv Wiki · GitHub
Install the current version of python3
```
pyenv install 3.7.0
```
Set the global python version
```
pyenv global 3.7.0
```
Run pyenv rehash (as indicated in pyenv’s help)
```
pyenv rehash
```
NO NEED TO install virtualenv & virtualenvwrapper in python3!
so the following two commands were NOT run:
```
#pip install virtualenv
#pip install virtualenvwrapper
```
upgrade pip:
```
sudo pip install --upgrade pip
```

Set up a syspip so that pip will not work unless a virtualenv is active

export PIP_REQUIRE_VIRTUALENV=true
#define a "global pip/system pip” function to use outside virtualenv:
syspip(){
    PIP_REQUIRE_VIRTUALENV="" pip "$@"
}

Create two virtualenvs: one for jupyter & one for generic tools

pyenv virtualenv 3.7.0 jupyter3
#pyenv virtualenv 3.7.0 tools3

Install jupyter in the respective virtualenv:

pyenv activate jupyter3
pip install jupyter

Test it :

cd ~/venvs # this is only just in case a notebook is saved
jupyter notebook

and get out of the virtualenv

pyenv deactivate

Set up the «jupyter3» & «tools3» environments to be available on the path (without requiring activation)
```
pyenv global 3.7.0 jupyter3 tools3
```
Test that jupyter works without activating the virtualenv
```
jupyter notebook
```

Check the paths for the various «pyenvs»

which python
which jupyter
which ipython

Or to see the full “real” path:

pyenv which python
pyenv which jupyter
pyenv which ipython

FINALLY: uncomment the last line (added above) to the bashrc:
```
pyenv virtualenvwrapper_lazy
```
and reload bash / restart terminal

Python Setup: updating process¶

[2018-10-24]

To update Python¶

First remove the requirement for pip to require a virtualenv: PIP_REQUIRE_VIRTUALENV=""
Then install the new python:
- Technically this should only be pyenv install 3.7.0
  but it didn't work so I had to use the following:
  CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install -v 3.7.0
  (something to do with Xcode Command Line tools no longer installing needed headers in /include)
Then restart the bash/shell session (so that the necessary first-run scripts are run)
Then run pyenv rehash (as recommended in the docs)
Next, set the new python interpreter as the global one: pyenv global 3.7.0 This overrides previous global interpreter settings
Just to be sure, check that the new one is set as the global python: pyenv global
Update pip: pip install --upgrade pip
Set the 'syspip' flag back on PIP_REQUIRE_VIRTUALENV=true

To update the python version in a pyenv-mananged `virtualenv`¶

There is no way to update the virtualenvs' python interpreter. So the process is:

Remove the virutalenvs¶

Remove virtualenv called 'jupyter3': pyenv uninstall jupyter3
Also removed the other one 'tools3': pyenv uninstall tools3
To confirm that the virtualenvs were removed, run pyenv virtualenvs to see the list of virtualenvs without the ones just removed.

Recreate the virtualenvs¶

Create a new virtualenv called jupyter3: pyenv virtualenv 3.7.0 jupyter3
Run pyenv rehash (per recommendations)
(Just in case, go to the virtualenvs directory: cd venvs)
Activate the new virtualenv pyenv activate jupyter3
Run pip list to see if pip needs updating (it did)
Update pip if necessary: pip install --upgrade pip
Install jupyter: pip install jupyter
Deactivate the virtualenv: pyenv deactivate

To set jupyter to be able to run from anywhere, set it as global: pyenv global 3.7.0 jupyter3

Test it:¶

cd ~/venvs # this is only just in case a notebook is saved
jupyter notebook

Virtualenvs with a lot of packages¶

If a lot of packages are installed in a virtualenv, a simple way to keep the same list of packages & installing them in one go:

To keep the list of packages installed in a virtualenv, epxort the list of packages using pip to a requirements.txt file & then install that list in the new virtual environment:

Export the list: pip freeze > requirements.txt
Create new virtualenv.
Activate teh virtualenv.
Install the list in the new virtualenv: pip install -r requirements.txt

Other useful notes¶

In macOS Mojave, installing python 3.7.0 failed
The solution:
- REF: Install failed, "zlib not available" on macOS Mojave · Issue #1219 · pyenv/pyenv · GitHub
- Code used:
  CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install -v 3.7.0