To Python or Not to Python?

I’m presuming you’re here because you’re a coder or you want to be one.
So, when someone asks you, “What is your favorite programming language?” you quickly answer, “Python!!” Does this describe you? You’re not alone! I’m with you. In reality, according to a 2020 study performed by GitHub based on the top searched programming language on Google, Python is the most popular language.

Why is Python so popular?

There are several reasons why Python is so popular, most notably the large amount of open-source libraries, community, forums such as stackoverflow, and so on, but the major reason is the simplicity with which it can be learned. So, someone who does not have a background in computer science or programming wants to design software or do data analytics. This is a question I am frequently asked “I’d like to work in data science, but I’m not familiar with Python! How long would it take me to master Python?” To be quite honest, the learning curve is exceedingly rapid, if not the fastest of any programming language. This is because Python was designed to be a general-purpose language, similar to writing English rather than coding. The goal was to create a language in which individuals could easily transfer their creative ideas into usable goods rather than wasting time attempting to construct complicated methods and boilerplate code that serves no value! And, fact, Python thrives in this area.

With Great Power Comes Great Responsibility!

As previously said, Python was designed as an English-like general-purpose language with commands that appear to be how you would ordinarily speak about it. In actuality, Python is really a wrapper over the popular statically typed language “C.” That is, whatever Python appears to be, is actually C-code operating behind the hood. This is due to the fact that C is one of the four Peta-FLOPS (one of the fastest) languages, which implies it can process more than a Peta (quadrillion or 10^15) FLoating point OPerations per Second. That is quite a lot. C has been benchmarked to be more than 70 times quicker than Python.

Wait! What?

That is correct. Although C is faster, it is not as user-friendly as Python. I mean, most “programmers” have done some C coding, but they would all agree that there is a lot of boilerplate code and a lot of stuff you need to be aware of and take care of. Python removes everything on its own. To code, you do not need to understand all of the internal workings at the register (hardware) level. Any competent online course could educate someone with no prior programming expertise how to code in about three months. However, this is not the case with C. It takes a lot of effort and practice to build effective and production-ready C code.

All that glitters is not Gold

Although Python is simple to learn and use, it is not necessarily the most efficient programming language. In reality, I just attended a Robotics conference where C++ and Rust were the most popular programming languages. Python, in fact, loses greatly in terms of scalability. When it comes to processing vast amounts of big data, most organizations dealing with such massive amounts of data, such as Google, Facebook, and others, have written all of their key modules in C, C++, and similar languages. This is proportional to C’s and its variations’ processing speed.

But why NOT Python?

Okay. Since you requested it, we will investigate what happens behind the hood and where Python fails to catch the train!!

In a statically typed language such as C, C++, Java, or others, all code begins with an explicit datatype declaration, but not in Python. A datatype is a class that defines the type of data that a certain variable should have. Python has a dynamic datatype allocation feature. This implies that the Python interpreter determines the datatype of a variable based on the content of the variable. Because it is not a fixed datatype, it is also liable to change on the fly. This is one of Python’s most impressive achievements. However, underneath this accomplishment comes the dark horror story of Python internals, which make Python one of the slowest languages. Let us attempt to comprehend this more thoroughly.

The Int datatype

Let us try to take a simple integer variable and try to understand how it is implemented in Python in retrospect of it’s C implementation.

In C, a variable can be declared of integer type by just specifying “int” keyword and the C compiler allocates 4 bytes (32 bits) of storage in the memory. In other words, the variable can store values from -2147483648 to 2147483647 both inclusive. This can be achieved by just one line of C code:

int var = 10;

In Python, the same thing can be done by directly assigning the value as:

var = 10

That’s all! At least, that’s how it appears on the surface.

But in reality, since Python itself is just a wrapper around a lot of C code. The Integer datatype in Python 3.4 is composed of the following code:

struct _longobject {
    long ob_refcnt;
    PyTypeObject *ob_type;
    size_t ob_size;
    long ob_digit[1];
};

What is this thing anyway?

Well that is a lot of C code for a simple thing! Why you asked? Let’s break it down then.

Python Integer Actual Representation in C

A Python Integer is effectively a long object (“_longobject”), which means it can hold a massive amount of data. It works by using a variable called “ob_refcnt” to dynamically allocate or delete internal memory storage for that variable. It has a pointer “ob_type” that indicates the type of the object. A variable “ob_size” that carries the variable’s size, and lastly a variable “ob_digit” that stores the actual value that we want to save in the variable. As we can see, in Python, we use a complicated structure that includes not only a pointer but also some additional variables for dynamic resource allocation and reallocation. This modular piece of code allows you to quickly alter the datatype of the variable, and the other members will modify to accept a different datatype as the same variable. But, as you may have suspected, this will not be the quickest or most efficient option. This will require more memory storage to retain all of the structure’s elements, and the access will be slower than a straightforward pointer reference in case C.

What about other datatypes and other data structures in Python?

Unfortunately, this is true for all of Python’s built-in datatypes. In fact, multi-pointer and variable sequences underpin practically the entirety of Python. All of this is done to reduce the amount of boilerplate code you must write! Hmmm??

What about lists and arrays in Python?

Glad you asked.

There is a distinction between the internals of lists and arrays. But first, let me clarify that while Python includes list as a built-in data structure, arrays are not included in the standard Python package. Since Python 3.3, the Array data structure has been included as part of the standard Python installation; however, the techniques for manipulating these arrays are provided by the package known as “NumPy” or “Numerical Python,” which substantially expands the possibilities of Python arrays.

Lists in Python
Lists in Python are mutable data structures, which means they may be updated at any moment and can combine many distinct data structures. That is, a list including numeric, text, boolean, and many more datatypes may be readily defined as:

myList = [10, 'abc', True, 0x07E]

We just looked at how the integer datatype is stored internally in Python, and other datatypes such as text, boolean, hexadecimal, and so on are stored in the same way. Every entry in a Python list is accounted for its own separate identity and is internally stored in a sophisticated C structure class. In other words, creating the preceding list myList would basically build a pointer which would point to four other distinct pointer object structures, each with around four elements inside to represent the variable’s value.

Arrays in Python
As previously stated, most efficient implementation of Arrays (or “ndarrays”) in Python are provided through the open-source package “Numpy”. Arrays are more efficient data structures in Python since they do not allow you to combine multiple data types within a single array. All the array elements needs to be homogeneous in nature. The array simply has one pointer, which links to a contiguous block of memory storage where the real data is stored. The Array structure also includes sophisticated characteristics such as dimensions, strides that define how the array is stored, and the type of contiguous memory sequence that must be allocated to the array. Arrays are therefore a far more restricted data structure than lists in Python, but they are also significantly superior. In truth, the majority of the operations included with the Numpy package are extremely well optimized and vectorized (can be handled in parallel), but that is a topic for another day.

Conclusion

Although Python has several shortcomings, particularly in terms of processing speed and scalability, it is one of the most user-friendly languages and has the strongest community support. Python is without a doubt the language of choice for creating any prototype to test any proof-of-concept without going through a rigorous development cycle. Regardless of its flaws, Python is prospering like no other language and is here to stay. So, if you’re debating whether or not to learn Python, go for it. Given Python’s quick development cycle, there is nothing more enjoyable than building anything in Python.

I hope this article gave you a clear and succinct overview of the numerous components of Python, including why it is so popular and its faults.
Give it some thought the next time someone asks you, “What is your favorite programming language?”

If you like this post, please follow me on Medium, where I routinely write stuff about Data Science, Machine Learning, and AI. Your follow request inspires small content providers like myself to generate more articles like these.

See you later in the next article!!