How to work with Python applications and modules in Yocto Project


This is a high-level tutorial and the intention is not to cover the Yocto Project in depth. If you are interested in detailed information, we recommend that you read the Yocto Project Mega-Manual.

Python is a commonly used language developing applications for the IoT and especially in the early stages of product development while one might still be evaluating an idea and maybe move on to a more performance oriented framework/languague once the idea has been validated

This tutorial will cover how to work with Python within the Yocto Project, and will cover how to:

  • include python3 in your image
  • inspect what Python modules are provided in the core layers
  • add a Python module that is not already provided
  • creating a recipe for a Python application

This tutorial will only cover Python3 as Python2 is reaching EOF by the end of 2019, but the workflow in Yocto Project is similar regardless if you are using Python2 or Python3.


To follow this tutorial, you will need:

Step 1 - Installing Python

Ensuring that python3 is installed in your image is relatively simple and can be done by adding the following to local.conf:

IMAGE_INSTALL_append = " python3"

NOTE You can add this to your custom image as well. Please take a look at the How to create custom images using Yocto Project tutorial.

With this addition we can now re-build our intended image and python3 will be included.

bitbake core-image-base

You should see something similar to below during your build process which hints that python3 package is being prepared:

0: python3-3.7.2-r0 do_configure - 6s (pid 19653)

But this will only install the python3 runtime on to our target, which in it self is not very useful unless we install some of the available Python modules and this is where it can get complicated to figure out what is available if you are not familiar with the Yocto Project environment.

Step 2 - Listing and installing Python modules

First of all there many modules that are provided in the standard Python standard library. Modules such as:

  • datetime
  • io
  • core
  • multiprocessing
  • gzip

The Yocto Project environment handles the standard library modules slightly different then one is used to on a desktop distribution, and as a beginner this can lead to confusion and most people wonder why the standard library modules are not installed on the device when you installed python3.

In the Yocto Project environment, the most common Python modules are split out in to individual packages so that you can select which parts of the standard library you are interested in. This is an optimization if you are building a minimalist system where you would typically would not want to include the complete standard library if the only module you are using is datetime or io.

You can get a list of these individual packages by reading this file:

cat ../sources/poky/meta/recipes-devtools/python/python3/python3-manifest.json

If the standard library module is not listed in the above file, it is part of the python3-misc package.

In the Yocto Project environment all module names are prefixed with python3-, and it looks like this (following our above example):

  • python3-datetime
  • python3-io
  • python3-core
  • python3-multiprocessing

NOTE! There is no python3-gzip package and that module is part of python3-misc package.

There is also a package called python3-modules which will install all standard library modules.

Thirdparty Python modules are provided as individual recipes, and one can run the following to get a list:

bitbake -s | grep ^python3

This list might be sparse and this is because only a small number of modules are provided by openembedded-core/poky.

To get a more complete list of modules we need to add the meta-openembedded/meta-python layer to our environment.

We can do this by cloning meta-openembedded:

NOTE You might already have the meta-openembedded layer in ../sources if you followed the tutorials on how to setup the Yocto Project environment. In which case you can skip this step.

git clone -b warrior ../sources/meta-openembedded

and then adding meta-openembedded/meta-python to our environment:

bitbake-layers add-layer ../sources/meta-openembedded/meta-python

As meta-python depends on meta-openembedded you may have to include that as well if you haven’t done so following a previous tutorial:

bitbake-layers add-layer ../sources/meta-openembedded/meta-oe

Now you can list available Python modules again by running:

bitbake -s | grep ^python3

You should see a significant increase in available modules.

Installing Python modules can be done in a similar fashion to how we installed python3, which was covered in Step 1 - Installing Python, e.g to install python3-requests we can add the following to local.conf:

IMAGE_INSTALL_append = " python3-requests"

It is also always a good idea to search the OpenEmbedded Layer Index
if you are looking for missing modules. In many cases you will find that someone else has created a recipe for it, and you can either add that layer to your environment or make a copy of the recipe.

Step 3 - Creating a recipe for a Python module

There is a high probably that you will run in to the situation where you application depends on a specific Python module which is not provided by the methods that we covered in the previous sections. This is simply because there is a wast number of third party Python modules and it has to provide everything.

In this situation you are forced to add a recipe for that specific module that you might require, but luck ally it is fairly simple due to the β€œframeworks” that the Yocto Project provides.

For the sake of this tutorial we will use the memory-profiler Python module as a example, which is not provided out-of-the-box.

Step 3.1 - Create a custom layer

To be able to proceed with this tutorial we first must create a custom layer that will contain any custom modifications or additions. Please note that the tutorials on this site will re-use this layer and if you have already created this structure by following another tutorial you can skip this step.

Create a new layer called meta-stargazer using the bitbake-layers helper application:

bitbake-layers create-layer ../sources/meta-stargazer

This will create a basic structure in meta-stargazer directory:

β”œβ”€β”€ conf
β”‚   └── layer.conf
β”œβ”€β”€ COPYING.MIT
β”œβ”€β”€ README
└── recipes-example
    └── example

3 directories, 4 files

Include the layer in our Yocto Project environment:

bitbake-layers add-layer ../sources/meta-stargazer

Step 3.2 - Create memory-profiler recipe

Create directory structure to house our recipe:

mkdir -p ../sources/meta-stargazer/recipes-devtools/python

Create the recipe:

cat <<- 'EOF' > ../sources/meta-stargazer/recipes-devtools/python/
SUMMARY = "This is a python module for monitoring memory consumption of a \
process as well as line-by-line analysis of memory consumption for python programs"
LICENSE = "BSD-3-Clause"
LIC_FILES_CHKSUM = "file://COPYING;md5=cde4ca348bb7feea790673481be0d980"

SRC_URI[md5sum] = "53ecb3d4be95a36fc1da339dce26d57f"
SRC_URI[sha256sum] = "5fa47b274c929dd2cbcd9190afb62fec110701251d2ac2d301caaf545c81afc1"

PYPI_PACKAGE = "memory_profiler"

inherit pypi setuptools3

RDEPENDS_${PN} += " \
    python3-psutil \

As you can see the recipe is fairly small and this due to the pypi and setuptools3 classes which do all the work for us, and what we essentially need to provide is the name and version of the module (license type + checksums) that we want to install and then it will download the sources from and create a package which we can include to our image.

Majority of the information that I provided in above recipe I got from reading the memory-profiler page on The most important to get right are:

  • module name
  • module version
  • dependencies
  • license type and md5sum
  • source md5sum and sha256

Now we can build the created Python module recipe:

bitbake python3-memory-profiler

Step 4 - Create recipe for a Python application

The workflow to create a recipe for a Python application is very similar to what we covered in Step 3 - Creating a recipe for a Python module. The difference might be that if you write a custom application it is probably not hosted but other then that the steps should be pretty much the same.

The most painful thing is usually figuring out which dependencies the application has and ensuring that the recipe specifies these to avoid using the python3-modules package which is usually the easy way out , but it does waste a lot of space.

Step 5 - Python pip

You can add Python pip to your image by adding the following to local.conf:

IMAGE_INSTALL_append = " python3-pip"

But this is only useful for experimenting and testing and the pip flow does not really apply in a Yocto Project setting where you at build time want to define the content of your distribution


In this tutorial we covered how to work with Python when using the Yocto Project in the hopes that it will help you get started quickly without need of digging to deep in Yocto Project internals. The layer that we created (meta-stargazer) and populated during this tutorial can be used a starting point for your own custom application/distribution layer.

If this tutorial was useful to you, please press like, or leave a thank you note to the contributor who put valuable time into this and made it available to you. It will be much appreciated!


Hi @mirzak fix

IMAGE_INSTALL_append = "python3"

in to

IMAGE_INSTALL_append = " python3"

Done, feel free to edit as well :slight_smile:

I remember doing this tutorial a few days ago and it worked. However, now, I worked on the stargazer image a little (basically following the systemd tutorial. When I add python3 again, start qemu and try to start python it fails:

root@qemux86-64:~# python3
[   56.914963] traps: python3[315] trap invalid opcode ip:7f7f73325686 sp:7ffe8025d930 error:0 in[7f7f731e2000+1bc000]
Illegal instruction

Do you have a hint on how to debug this?

Is it possible that this issue is related to the host system? Python3 works perfectly fine when building for qemu on Ubuntu 18.04.3 LTS but if fails on when building on another machine running Ubuntu 18.04.4 LTS. Could you check if anyone is running 18.04.4 LTS to exclude this possible error source? Would be very appreciated.

EDIT: The most likely reason was most obvious: A mismatch between poky (warrior) and meta-openembedded (zeus). I stashed everything and just started a clean, fresh rebuild. If the error persists, I’ll keep you updated.

EDIT II: I was wrong. The error persists on a clean warrior build, with and without meta-oe/meta-python from the same branch.

Hi @HerrMuellerluedensch, I think this error depends on how you start the QEMU machine.

Can you try adding -m core2duo to your qemu arguments on startup?

This advice did not work for me,it was close however. Instead I used

-cpu core2duo

On my host the -m flag is for memory of the virtual host