Start building your own chatbot now!

Every September, Artificial Intelligence (AI) enthusiasts meet up in San Jose to present and discuss the latest developments in the field. O’Reilly’s Artificial Intelligence Conference is the AI-focused counterpart to the organizer’s data engineering Strata Data Conference. And next year, both events will be colocated in San Jose. My visit to this year’s Artificial Intelligence Conference not only provided insight into hot topics and key projects; it also showed how AI is maturing as an engineering discipline. I’d like to take this opportunity to share some of my impressions from the conference with you.

As soon as I entered the conference venue, it was clear that, in this year’s iteration, the focus wouldn’t be on algorithms, but on supporting them: model life cycle and infrastructure, ethics, and explainability. While algorithms are at the core, they are brought to life by a multitude of techniques and tools around them. This can be visualized perfectly using the famous illustration from Sculley et al.’s Hidden Technical Debt in Machine Learning Systems paper:



In addition, the overarching theme of ethical AI connected every step of the machine learning workflow, taking center stage in many presentations. With these preliminaries out of the way, let’s take a walk through the event together.

Be Ethical

Roughly a quarter of this year’s keynotes focused on one topic: ethical development of AI. Microsoft and Dataiku shared their views on how to guarantee a responsible workflow for creating AI solutions and also open-sourced techniques for fair learning algorithms and for creating interpretable models. Google went a step further and not only shared their guideline on responsible AI practices, but also proposed a framework of model cards designed to deliver invaluable background information on how a machine learning model came into being.



These cards summarize topics and questions that should be discussed and documented during model development to enable the parties that receive the model to apply it in the right circumstances and draw attention to potential pitfalls. This approach brings structure and standardization to the metadata attached to model deliveries, an area that has received little attention to date.

Explainability: Greater Transparency, Greater Trust

The explainability of machine learning decisions was presented as a current trend in the ethics of AI. Explainability not only facilitates development and debugging during the creation of algorithms and models; it also delivers much-needed transparency. The Institute for Ethical AI & ML presented XAI, a machine learning framework that supports explainability from the ground up, to enable data scientists and domain experts to more easily understand the “Why?” behind a decisions derived from the output of an ML system. This is a prerequisite for bolstering end-users’ trust in any kind of intelligent system.

Ethics and its role in machine learning were at the heart of many discussions, even beyond the keynotes, and especially in executive-level briefings. It was striking to see this important topic being taken up everywhere, even (quite literally!) on the street:

Finally, ethics also has an impact on how the AI scene does business. For a primer on the potential ethical pitfalls of creating AI in the language space, (for example, chatbots), check out our previous post on the topic.

Build All the Things!

Now let’s dig a little deeper into the technical aspects. I want to start by talking about some big trends. First, automated machine learning: As I walked across the show floor, there was evidently no shortage of startups like Snark.AI, Augur.AI, Capice.Cloud, or tackling the topic of automated ML at different stages and in different ways. Their common goal is to expand the AutoML space and democratize machine learning by removing the need for data science expertise to solve business problems – just bring your data and get your model. Sounds easy: But it’s important to be on the lookout for marketing-speak when approaching this topic.

Even the solutions mentioned above require abundant high-quality training data. So, continuing our walk through the show, we come to the next cohort of companies working on solving the data acquisition problem. Players such as or Stanford’s Snorkel advertise solutions designed to facilitate data collection and annotation, whether manually or with the help of weak supervision.

Moving ML Solutions out of the Lab and into Production

Now that we have enough data and a complete ML model, we just need to put that model into production – which brings us to the third big trend: deployment and management of ML solutions. The number of lectures on deploying machine learning models reached double-digit territory this year. A key quote here is: “The model you want to train is not the model you want in production.” In other words, a model that performs well in a lab environment must be optimized to fulfill production requirements, such as low-latency inference, dynamic scaling, monitoring, and resource savviness.

To tackle this challenge and create the proper infrastructure, Google open-sourced TensorFlow Extended. This is the tech giant’s way of putting machine learning use cases into production on top of TensorFlow (TF). Alternatively, if TF is not the right choice, the Kubeflow community has got you covered. This toolkit runs ML tasks on top of Kubernetes, brings together many lower-level components, and allows a best-of-breed design approach throughout the ML pipeline. Getting started could hardly be easier – just head over to and deploy your first system on Google Cloud Platform (GCP). And where GCP is involved, Microsoft Azure is never far away: Microsoft shared its ML reference architecture for running and deploying models on Azure.

Optimizing Models for Real-World Deployment

With the infrastructure in place, we can look at optimizing our models to save us some computation and money. Optimization techniques like knowledge distillation, model pruning, or quantization are widely used to strip unnecessary baggage from the models created in the lab. In the area of natural language understanding (NLU), Intel’s AI Lab open-sourced their solution for quantizing BERT and distilling knowledge for transformer models pre-trained on huge datasets. This work is part of the greater NLP Architect project, which offers insights into cutting-edge deep learning approaches for NLU. Similarly, Microsoft open-sourced their way of working with natural language in their best practices for NLP repository.

Wrapping Up

As we come to the end of our tour of the conference, the focus of the 2019 event is clear: Create simpler methods to develop and deploy AI. Make AI a ubiquitous tool for all use cases. Propel AI engineering out of the lab and into production. And all of this while keeping track of the impact on the humans that AI is meant to support. Incidentally, these goals fully align with our mission at SAP Conversational AI: We build the platform for conversational agents to easily tackle language understanding challenges and bring AI into every aspect of your product – without the need for data science expertise or AI deployment skills, but with a clear commitment towards ethics.

One more thing: Beyond the scope of current algorithmic research, Prof. Kenneth Stanley presented his ideas on open-ended algorithms: an approach where it’s not about finding a solution for a given problem. Instead, open-ended algorithms try to find problems to solve. It’s worth a read if you’re interested in finding out what lies ahead.

There are currently no comments.

This site uses Akismet to reduce spam. Learn how your comment data is processed.