0800 808 5270

Latest News

Latest News on Cyber, Penetration Testing, BPM & Digital development
python

Time to shed Python 2

python

Don’t constrict yourself, Python 2 slithers off into the sunset in 2020.

The end of life (EOL) date for Python 2 has been a long time coming, but it’s finally in sight. As of the 1st of January 2020, Python 2 will no longer be supported. There will be no more bug fixes, or security updates, from Python’s core developers.

So, if you’re still using 2.x, it’s time to port your code to Python 3. If you continue to use unsupported modules, you are risking the security of your organisation and data, as vulnerabilities will sooner or later appear which nobody is fixing.

Scale of the problem

Below is a table of ten popular Python packages and their stats from the Python Package Index, a repository of software developed and shared by the Python community.

Each of the packages in the table is downloaded millions of times per month. In this case, the data is from June 2019.

As you can see, most of the package downloads are still for Python 2.x versions. In the best case, the web application framework Flask is downloaded mostly for 3.x. But even there, nearly a third of the downloads are still for 2.x.

Even if only a portion of these downloads are being used in live projects, the Python 2 EOL could potentially affect the security of millions of systems.

Package
Python 2
Python 3
Unknown Version
Colorama 82.07% 17.71% 0.22%
botocore 72.86% 26.84% 0.30%
urllib3 64.14% 35.50% 0.37%
Requests 53.37% 46.02% 0.61%
scikit-learn 43.80% 55.90% 0.30%
NumPy 40.69% 58.70% 0.61%
Pillow 37.78% 61.31% 0.91%
TensorFlow 37.66% 61.25% 1.09%
Matplotlib 33.17% 66.34% 0.49%
Flask 31.28% 68.33% 0.39%

Issues caused by the demise of Python 2

There are many issues brought about by the end of life of Python 2, here are some you may encounter.

Dependencies

Many popular projects such as NumPyRequests, and TensorFlowhave pledged to drop support for 2.x by 2020 and some already have.

This means that if you want to use the latest features of your favourite modules, you’ll need to be using Python 3. The longer you wait to update, the more the Python 3 versions of your dependencies will have changed, and the more difficult updating will become.

You may be holding other developers back

If you maintain a library that other developers depend on, you may be preventing them from updating to 3. By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others.

You may not publish any code outside of your organisation but consider your colleagues who may also be using your code internally.

You are missing out on the latest features

Python 3 has a multitude of new features you are missing out on, including:

  • The yield from expression – allow a generator to delegate part of its operations to another generator.
  • Unicode strings – Unicode is easier to handle.
  • The print function – the print function has extra features that make it more flexible.
  • Views and iterators instead of lists – some well-known APIs no longer return lists. For example, dictionaries return views of the keys, values, or both.
  • The ‘multi-with’ statement – complex with statements are easier to read.
  • Unpacking using * and ** – extended uses of the * iterable unpacking operator and ** dictionary unpacking operator. It is now possible to use an arbitrary number of unpacking operators in function calls.
  • Keyword-only arguments – allow arguments to be present after a varargs argument.
  • F-strings – a new kind of string literal evaluated at runtime that can contain any valid Python expression.
  • Lots and lots of speed-ups and optimisations.

When Python 3.0 was released, some of the new features were backported into Python 2.7 and may be familiar. If you have been stuck using version 2.6 or below then these may be new to you.

Help for developers

Porting Python 2.x code to Python 3 can be quite a daunting process. Fortunately, there are tools and resources available to make it easier.

Can I Use Python 3 is a program that checks your project dependencies to see if any are preventing you from using Python 3. Use this program to see if you need to swap out any libraries stuck on Python 2.x. Where this is the case, it may be worth visiting the library homepage to see if there are plans to port to 3.

2to3 is a Python program that attempts to convert 2.x source code into 3. It is usually installed with the Python interpreter as a script. You can run this program without writing any changes, if you just want to see a diff of each change it will make to your source code. Note that this isn’t perfect, you may still have to fix some code manually.

Supporting Python 3: An in-depth guide is a free, open source eBook that guides you through the process of adding Python 3 support. The book explains some common migration problems you might encounter and lists ways you can improve your source code using the new features in Python 3.

Six is a Python 2 and 3 compatibility library that provides developers a way to write code that is compatible with both Python 2 and 3. Using this library, developers have a choice in what version of the Python interpreter they use to run your code. If you decide to use this library, make sure your code accounts for the fact that the Python version number may increase to 4.0 soon (at the time of writing the latest version is 3.7.4).

Use the experiences of others to help you – lots of people have posted their experience of porting their codebase. Dropbox migrated their codebase to Python 3 in 2018 and wrote a blog post detailing their experience.

The Python website also has documentation that you may find helpful.

When migration is not an option

If migrating your code base to Python 3 is not possible, another option is to pay a commercial company to support Python 2 for you.

At least one company has already announced a support package for Python 2 and Python 2 third-party packages.

Some Linux distributions contain Python 2 and will be supported past its end of life. For example, CentOS 7 and Debian 10 both use Python 2 and will be supported into 2024 but it is unknown what this support will look like.

An opportunity to improve

Maintenance is an important element of any Software Development Lifecycle (SDLC). Part of this step is improving the performance of the software and enhancing security, both of which can be achieved by upgrading to Python 3.

In other words, if you are still using Python 2, this is an opportunity to improve how you manage your software dependencies and minimise your security debt.

Don’t ignore the risks

At the NCSC we are always stressing the importance of patching. It’s not always easy, but patching is one of the most fundamental things you can do to secure your technology.

The WannaCry ransomware provides a classic example of what can happen if you run unsupported software. It infected more than 230,000 computers, causing major disruption around the globe. More recently, the Equifax breach has resulted in a settlement of up to $700 million.

By making the decision to continue using Python 2 past its end of life, you are accepting all the risks that come with using unsupported software, while knowing that a secure version is available.

Food for thought

Hopefully, this blog post has helped to convince you that it’s time to start migrating to Python 3. If you’re still mulling your options, our risk management guidance, and secure development and deployment guidance, may help you make a decision.

Rich M

Platform Security Researcher

Source: National Cyber Security Centre

https://www.ncsc.gov.uk/blog-post/time-to-shed-python-2