For some its earlier, for some later, and some never get there at all.
Most developers with a long career do experience this point, though.
Ill call it thebuild-it-yourself point.
If youve arrived there yet, you know what the first questions are: How does it work?
What does the user experience?
How does the data flow?
And many more questions like this.
I wont answer these answers for you here.
Theyre highly specific to whichever project youre starting.
And every single one of these questions deserves at least one article of its own.
It’s free, every week, in your inbox.
I will answer one question though:Which language is best for the project?
You might be thinking that this is very project-specific, too.
And youre not totally mistaken.
But every programming language has a few pitfalls.
And Python, it turns out, has quite a lot of pitfalls.
Especially when youre trying to build a large program with it.
Variable declarations dont exist and thats a problem
TheZen of Pythonstates: Explicit is better than implicit.
But when it comes to variable declarations, implicit is way more common than explicit in Python.
The char is a throw in identifier and tells you that everything hereafter relates to a string.
The piece notpython is the name that Ive given to this string.
The [50] tells you that C will reserve 50 characters worth of memory space for this.
And, finally, a semicolon to end this neatly.
This kind of explicit declaration ismandatoryin C. The compiler will go on strike if you omit it!
This way of doing things seems silly and tedious at first.
But it pays off.
Compare that to Python.
There you pretty much invent variables as you go.
In Python, theres no way to understand what a variable is doing except digging right into the code.
Or if your project isnt very complex.
But with larger projects… Shit hits the fan.
you’re able to do explicit variable declarations in Python.
But only the most diligent programmers do that.
And when the compiler doesnt complain, many forget about additional lines of code like these altogether.
Coding Python is fast.
Reading Python is easy, for small and simple projects.
Oh module, where do you belong?
If you thought that things cant get much worse, youre wrong.
The question where a variable starts living in your code doesnt stem from implicit declarations only.
Variables may come fromother modulesas well.
This is annoying because there are more modules than the ones it’s possible for you to find onPyPI.
it’s possible for you to also import any other Python files on your machine.
So quickly googling your function or variable wont always help.
But it gets even worse.
Modules can depend on other modules.
Whats worse, sometimes its not such a simple tree.
Its a labyrinth.Dependency hellis a real thing, coined by Pythonians.
Circular dependenciesare the ugliest beast in the labyrinth.
If module A depends on module B, but B also uses parts of module A ouch.
Not a big deal in small projects.
But in big ones… welcome to the jungle.
Dependency collisions en masse
Oh, Im still not finished with my rant on modules.
Its not only the modules themselves, but also their versions.
In principle its great that Python has such an active user base and many modules are updated regularly.
Theres just one problem: not all versions of a module are always compatible with other modules.
Say for example youre using modules A and B.
You dont care about C. You only want A and B.
No tool in the worldis going to help you with this conflict.
If youre lucky, youll find a patch written by someone who has encountered the same problem as you.
If youre not so lucky, youre going to have to write the patch.
Or you use a different package.
In any case, youre going to need extra time for this.
Its a jungle, and youll need patience and some tools to navigate it.
Dependency collisions aside, there are some nice tools around.
Theres pip that makes it easy toinstall packages.
Andvirtual environmentskeep all packages in one place and apart from your main Python installation.
For bigger and messier projects, theres also conda, YAML files, and more.
But youll need to learn how to use each tool anyways.
And youll need to spend a minimum amount of time dealing with these problems.
Do newborn horses run at all?
Anyway, back to Python.
Tools like pip, requirements.txt and virtual environments will help you navigate mild forms of dependency hell.
On every new machine youll need to check and potentially reinstall each single requirement and its version.
The only really portable solutions areJupyter notebooks.
Here it’s possible for you to write things in any version you like.
But with graphic interfaces its quite difficult to handle large projects with many interlinked files.
Maybe thats why Ive never seen a large project in Jupyter notebooks.
Even though they surely exist.
Other languagesjust have virtual machines.
All of which are slightly more clumsy to handle than a virtual machine.
But hey, at least they work.
There are many advantages to this: C packages might not exist in Python, and areusually faster.
Scientific packages often exist only in Fortran forlegacy reasons.
In effect, youre going to have to use compilers like gcc, gfortran, and perhaps others more.
And thats a hassle!
And the documentation for Fortranisnt that much shorter either.
Building your whole project in C might be slower to code at first.
But youll prevent situations where you have to mess around with multiple compilers and interfaces.
C is so old that theres packages for almost anything.
Evenuser-friendly machine learning packages.
It made memory management incredibly easy for the end user.
Compare that to C where you literally reserve bits of memory for every single variable!
Basically, the GIL counts how many times a variable has been referenced in every section of the code.
If the variable is no longer needed, then it frees the memory space it occupies.
In small projects, the GIL helps with performance-boosting because unnecessary memory space is wiped out.
But in bigger projects theres a problem: the GIL doesnt likemultithreading.
Theres just one little problem: the GIL only works on one thread at a time.
It just depends where the GIL happens to be at the time.
This can lead to very weird bugs, as you might imagine…
There are workarounds for this, but they are allnot very pretty.
As an alternative, theres multiprocessing.
But it generallywont be as fastas multithreading inlanguages without a GIL.
Concurrency and parallelism are still clunky and messy
Weve already seen one downside of concurrency.
When youre doingmultithreading, the global interpreter lock can slow things down.
Or cause weird errors.
The same downside applies to Pythonscoroutines.
Both of them are implementations of concurrency.
Instead of letting the computer sit idly by, coroutines assign another task to it.
Streaming data could be named as an example.
Multiprocessingis your best friend then.
It basically tells the computer to use multiple cores and save time.
All three techniques, threading, coroutines, and multiprocessing, face similar problems though.
Theyre not that hard to implement in Python.
But the code looks clunky and is hard to read, especially for beginners.
But if you are, you might want to consider your options.
What to use instead of Python
Python isnt all evil.
But it has its downsides.
If you want something thats portable to any machine, then Java, Clojure or Scala are great options.
In the beginning theyre harder to learn than Python, but the time you invest pays off.
And you could always combine languages.
Python is great for quick scripting, drafting, and even medium-sized projects.
This makes the code execute quicker, and you still get to enjoy the advantages that Python gives you.
In big projects, Python isnt forbidden.
But it might not be the only language thats used.
If youve already reached yourbuild-it-yourself point, remember that no one language is the Holy Grail.
Despite its downsides, Python is cool and convenient.
you’re free to always get around the pain points by integrating code in other languages.
A big thank you toJanek Schleicherfor inspiring me to write this story.
This article was originally published on Medium.
you’re free to read ithere.
