Updating Python in Companion

Is Python 3.7 and up supported on this companion version?

Hi @farhang,

Companion 0.0.28 still uses Raspbian Jessie, so the default Python 3 install is 3.4. That doesn’t stop you from installing a later version separately if you want to, but be aware it’s a few steps and takes a while for it to compile. I’ve provided instructions below if you’re interested in doing so.

As a timing indication, expect the full process to take ~30-60 minutes.

Requires Internet Connection

Check that your companion computer is connected to the internet. On the companion web interface Network page your “Internet Status” should say connected.

Connect to the Raspberry Pi

You can either use the terminal over browser functionality, or you can ssh in (default password is companion).

Stop Companion Functionality

Compiling code wants all the CPU it can get - close the non-essential screen sessions to allow compiling to have more resources. Keep the web-related sessions open to allow webterminal use, and monitoring CPU usage and temperature with companion web interface.

for SESSION in $(screen -ls | grep "(" | cut -f2 | grep -v web)
do
    screen -S $SESSION -X quit
done

Update Raspbian

Get the latest list of packages. Includes the main backport forJessie, so a suitable SSL can be installed.

# add the required key for jessie archives
gpg --keyserver keyserver.ubuntu.com --recv-key 7638D0442B90D010
gpg -a --export 7638D0442B90D010 | sudo apt-key add -
# add a source for the 'main' jessie backport
echo "deb http://archive.debian.org/debian/ jessie-backports main" | sudo tee /etc/apt/sources.list.d/jessie-backports.list
# ensure apt is pointing to a valid url for the required Raspbian sources
sudo sed -i 's/mirrordirector/legacy/g' /etc/apt/sources.list
# get the list of available packages, ignoring the expired jessie release-key
sudo apt -o Acquire::Check-Valid-Until=false update

Install Prerequisites

Installing Python on Linux is much smoother if some dependencies are installed first

sudo apt install -y build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev libffi-dev uuid-dev lzma-dev
# required for SSL/pip to work
sudo apt install -t jessie-backports libssl-dev=1.0.2l-1~bpo8+1

Download and Extract Python

# specify which version you want to install
VERSION=3.8.11
# download it
wget https://www.python.org/ftp/python/${VERSION}/Python-${VERSION}.tgz
# uncompress it
sudo tar zxf Python-${VERSION}.tgz

Make and Install

# enter the source folder
cd Python-${VERSION}
# configure for building (checks for prerequisites, takes a little while) 
sudo ./configure --enable-optimizations
# build the source - expected to be the longest step
# !WARNING! if RPi is enclosed use -j2 or no -j flag to avoid overheating/throttling
sudo make -j4
# install as an 'alternative' (doesn't overwrite existing python3)
# If you want to overwrite it you can do `sudo make install` instead,
#   but that may cause issues with existing libraries not being carried 
#   over - I haven't tried it
sudo make altinstall

Confirm working

The new python install should be available by the major version number. You can check that it’s working by running python3.x -V and seeing if the top line is what you tried to install (e.g. for python 3.8.11 you can run python3.8 -V).

Note there may be a bunch of messages like profiling:...: Cannot open - these are related to the source and will be removed in the next step.

Clean up

# exit the Python source directory
cd ..
# remove the compressed and extracted Python source
sudo rm -rf Python-${VERSION}*

Usage

  • From here you can run code using your new Python version by calling it with the major version number (e.g. python3.8 my_script.py).
  • If you prefer executing scripts directly you can set the shebang to #! /usr/local/bin/python3.8 (or whichever version you’re using)
  • For installing libraries pip is also accessible via the major version number (e.g. pip3.8 install ...), or you can use python3.8 -m pip install ....

The normal companion functionality will be available again after restarting the raspberry pi.

1 Like

Thanks alot @EliotBR . Currently we are using a branch which was given to me from Bluerobotics.
I will try this soon which no idea whan it would be.

@EliotBR
This took a bit of time. I tried it on the latest companion and apt update is giving the below output:

Hit http://archive.raspberrypi.org jessie InRelease
Ign http://mirrordirector.raspbian.org jessie InRelease
Ign http://mirrordirector.raspbian.org jessie Release.gpg                               
Ign http://mirrordirector.raspbian.org jessie Release          
Hit http://archive.raspberrypi.org jessie/main armhf Packages
Hit http://archive.raspberrypi.org jessie/ui armhf Packages                          
Ign http://archive.raspberrypi.org jessie/main Translation-en_GB                                    
Ign http://archive.raspberrypi.org jessie/main Translation-en                         
Ign http://archive.raspberrypi.org jessie/ui Translation-en_GB
Ign http://archive.raspberrypi.org jessie/ui Translation-en
Err http://mirrordirector.raspbian.org jessie/main armhf Packages                                                                                                                                                 
  404  Not Found [IP: 93.93.128.193 80]
Err http://mirrordirector.raspbian.org jessie/contrib armhf Packages                                                                                                                                              
  404  Not Found [IP: 93.93.128.193 80]
Err http://mirrordirector.raspbian.org jessie/non-free armhf Packages                                                                                                                                             
  404  Not Found [IP: 93.93.128.193 80]
Err http://mirrordirector.raspbian.org jessie/rpi armhf Packages                                                                                                                                                  
  404  Not Found [IP: 93.93.128.193 80]
Ign http://mirrordirector.raspbian.org jessie/contrib Translation-en_GB                                                                                                                                           
Ign http://mirrordirector.raspbian.org jessie/contrib Translation-en                                                                                                                                              
Ign http://mirrordirector.raspbian.org jessie/main Translation-en_GB                                                                                                                                              
Ign http://mirrordirector.raspbian.org jessie/main Translation-en                                                                                                                                                 
Ign http://mirrordirector.raspbian.org jessie/non-free Translation-en_GB                                                                                                                                          
Ign http://mirrordirector.raspbian.org jessie/non-free Translation-en                                                                                                                                             
Ign http://mirrordirector.raspbian.org jessie/rpi Translation-en_GB                                                                                                                                               
Ign http://mirrordirector.raspbian.org jessie/rpi Translation-en                                                                                                                                                  
W: Failed to fetch http://mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages  404  Not Found [IP: 93.93.128.193 80]                                                                     

W: Failed to fetch http://mirrordirector.raspbian.org/raspbian/dists/jessie/contrib/binary-armhf/Packages  404  Not Found [IP: 93.93.128.193 80]

W: Failed to fetch http://mirrordirector.raspbian.org/raspbian/dists/jessie/non-free/binary-armhf/Packages  404  Not Found [IP: 93.93.128.193 80]

W: Failed to fetch http://mirrordirector.raspbian.org/raspbian/dists/jessie/rpi/binary-armhf/Packages  404  Not Found [IP: 93.93.128.193 80]

E: Some index files failed to download. They have been ignored, or old ones used instead

Do you have any input on it?

I fixed it in sources list. Ok now. Will try your solutuon after.

We’ve just realised this apt update problem is a broader issue (affects basically everyone), so we’ll be making an update in the next couple of days to fix it. Anyone running into it at the moment can get apt working again by running

sudo sed -i 's/mirrordirector/legacy/g' /etc/apt/sources.list

in the web terminal.

1 Like

I have this now:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

running build_scripts
creating build/scripts-3.8
copying and adjusting /home/pi/sources/Python-3.8.11/Tools/scripts/pydoc3 -> build/scripts-3.8
copying and adjusting /home/pi/sources/Python-3.8.11/Tools/scripts/idle3 -> build/scripts-3.8
copying and adjusting /home/pi/sources/Python-3.8.11/Tools/scripts/2to3 -> build/scripts-3.8
changing mode of build/scripts-3.8/pydoc3 from 644 to 755
changing mode of build/scripts-3.8/idle3 from 644 to 755
changing mode of build/scripts-3.8/2to3 from 644 to 755
renaming build/scripts-3.8/pydoc3 to build/scripts-3.8/pydoc3.8
renaming build/scripts-3.8/idle3 to build/scripts-3.8/idle3.8
renaming build/scripts-3.8/2to3 to build/scripts-3.8/2to3-3.8
* Error in `./python': double free or corruption (!prev): 0x016ef870 *
Aborted
Makefile:617: recipe for target 'sharedmods' failed
make[3]: * [sharedmods] Error 134
make[3]: Leaving directory '/home/pi/sources/Python-3.8.11'
Makefile:510: recipe for target 'build_all_generate_profile' failed
make[2]: * [build_all_generate_profile] Error 2
make[2]: Leaving directory '/home/pi/sources/Python-3.8.11'
Makefile:484: recipe for target 'profile-gen-stamp' failed
make[1]: * [profile-gen-stamp] Error 2
make[1]: Leaving directory '/home/pi/sources/Python-3.8.11'
Makefile:495: recipe for target 'profile-run-stamp' failed
make: * [profile-run-stamp] Error 2

Given the error message at the top:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

did you remember to install the prerequisites with apt? libssl-dev is one of them, and if that’s not updated then the version on your RPi could be outdated.

EDIT: just tried this myself, and it seems like the latest available libssl for raspbian jessie is too old for this. I continued the installation without it, which works fine, but ssl is used when pip installing from PyPI so installing libraries would be more of a pain (you’d need to download the wheels yourself and then install from that).

I’ve found this blog post about upgrading the ssl with a back-ported later version, but having trouble getting it working at the moment. Will post back when I get something working, or if I’m not able to.

I’ve updated the instructions with the required changes. Since jessie is no longer maintained it’s only stored in the debian archive, and the keys/certificates for it are expired. That adds a few steps to the instructions, but it should at least work properly to install packages now.

Note that we’re working on a major companion update at the moment which we’re planning to release within the next few months. It’s pretty much a full rewrite, and should be much easier to modify and use across various systems. The default OS will be either Buster or Bullseye (I’m not sure which yet, and Bullseye isn’t yet released), but I believe the main components are dockerized and using Python 3.9 anyway.

1 Like