Python’s Hidden Ecosystem: The Quiet Force Behind Modern AI

Learning tech speak

When I started to find out about Python, I’d imagined it would just be another piece in the ‘how to build a custom GPT’ tech jigsaw puzzle.

But it’s turned out to be so much more. Aside from understanding what Python does, why it became so popular and the slightly quirky story of its naming (all explored in last week’s article), I gradually realised the Python story wasn’t just about being the “engine” behind so much of modern software and AI.

In many respects, Python’s story provides a glimpse into a tech world where major shifts and developments have been happening over many years.

These changes won’t ever make the evening news bulletins but they are probably more significant in their way, than many of the stories that do.

From data libraries to governance, funding and the role of big tech, the behind-the-scenes of Python is fascinating and inspiring.

The Origins of Python’s Data Libraries

Let’s start with the data libraries and their quirky sounding names.

  • There’s NumPy, which began life in a US university lab, created by researchers who simply needed a faster way to crunch numbers.
  • Pandas originated from a single developer, Wes McKinney, frustrated that nothing existed to handle time-series financial data elegantly.
  • Matplotlib, started when John Hunter, a neurobiologist, wanted publication-quality figures for his research papers.
  • Scikit-Learn, emerged from French research institutes and academic collaborations.

None of these libraries began as products. They weren’t built for profit. They were built because somebody, somewhere, had a real problem and happily shared the solution with others who might be interested.

That openness, that instinct to share, refine, and collaborate, is the part that surprised me most. These libraries became the backbone of data science not through corporate might, but through community momentum.

The Interplay with Big Tech

Of course, the silicon-valley giants do appear later in the story. Google released TensorFlow to the world. Meta (then Facebook) built PyTorch.

It would be easy to assume that these libraries are the most important and have a significance greater than their non-corporate counterparts.

But, in fact, you’d be wrong. Google and Meta open-sourced these libraries deliberately because the scientific, academic, and open-source communities had already set the pace. If they’d tried to keep these tools proprietary, they’d have been ignored. Big tech wasn’t defining the field. It was catching up with it.

Even more interestingly, PyTorch has now been handed over to the Linux Foundation, where it’s governed by a board of companies and independent contributors — a recognition that the ecosystem is bigger than any one organisation.

Behind the Scenes: Governance and Funding

As the PyTorch example illustrates, these libraries aren’t held together by goodwill alone. They’re supported by some surprisingly robust governance structures.

  • There’s NumFOCUS, a nonprofit that provides financial and administrative support for libraries like NumPy, Pandas, Matplotlib and others.
  • There are steering councils, community working groups, research grants and corporate sponsors all helping to shape the ecosystem.

I’d had no idea. But behind the scenes, there are thousands of contributors and a governance model that is, in its own quiet way, highly democratic.

Why This Matters for Me — and for L&D

What struck me most is how much of PerformaGo’s capabilities will rely on this ecosystem. An ecosystem built by people I’ll never meet, who have already created solutions for numerous features and functions that I’ll want to use and incorporate. I’m so grateful that I can be the beneficiary of their earlier work and generosity.

And I’m sure I won’t be the only one. Because as GPTs become more central to workplace learning, evaluation, and performance support, it’s these libraries that will give them their analytical muscle.

It’s these libraries that will allow “non-technical” people like me to work with data, generate insights, and build tools that would once have required much more technical know-how than ordinary folk could muster.

Discovering this ecosystem reminded me that we’re all building on foundations laid by others. And that sometimes, big leaps come from the most unexpected starting points.

Who could have imagined that a researcher wanting to produce a better-looking graph, or a developer not liking how spreadsheets handled dates would lead to the Python data libraries of today.

There’s something wonderfully serendipitous about that.

Until next time…

My Most Recent Diary Entries...

Diary Entries from the Last 3 Months...