Algorithmia is moving into the new year with a lot of great improvements to the platform. We are always looking to increase performance in production model serving and infrastructure efficiency in every release while making the day-to-day developer and data scientist experience even more seamless.
Projects for this release focused on providing greater flexibility and data connectivity options and enabling queuing for user calls. These updates are part of our dedication to continuously improve our platform for our users. We’ve made a number of changes under the hood that will provide tangible boosts to speed, flexibility, and supportability.
Image Per Algorithm
We are really happy to announce the release of a major evolution to the architecture of how algorithm containers are created and loaded when requested on the Algorithmia platform. We now enable more flexible instances and support new machine learning frameworks with many different combinations.
New framework releases will be quickly added to allow data scientists and developers to take advantage of the latest technologies available to them.
Image Per Algorithm changes how algorithm version container images are created and shifts the creation of containers to when the algorithm changes instead of when a request comes in for the algorithm and a slot needs to be loaded. This shift towards package sets enables the platform to iterate faster and support emerging technology, and it increases the performance of algorithm runtimes.
This allows the Algorithmia cluster to target package sets toward tailored scenarios, like running a specific version of Tensorflow with Python 3.7. There is a package set available that is pre-configured for that scenario that will work in GPU and CPU environments.
The Algorithmia team will be creating new package sets in future releases and over time will transition all class algorithm base templates to new Image Per Algorithm-based package sets.
New package sets will be released to the Hosted Service on Algorithmia first and then rolled up together in future quarterly releases for Algorithmia Enterprise.
If you are interested in any package sets that have been released to the Hosted Service, please let your Deployment Architect know to work on getting it imported before your next upgrade.
Initial Package Sets for This Release
This release includes the following package sets that can be used for newly created algorithms going forward. New package sets will be created and shipped to the Algorithmia Hosted Service and bundled into future releases of Algorithmia Enterprise.
We will enable Enterprise Admins to create custom package sets to help provide best practices and templates for their data science teams specific to their organization.
Algorithm Management APIs
Building on the previous release, we are expanding the available set of Management APIs that developers and system administrators can use to interact with the Algorithmia platform through automation. The main scenarios we heard customers want to automate with these newly available APIs are to enable continuous integration and continuous delivery pipelines.
We’re happy to say that the last two steps in an automated pipeline are no longer exclusive to the Algorithmia UI.
The additional APIs that have been added in this release are:
- Create algorithm
- Get algorithm information
- Update algorithm
- Publish algorithm
These algorithm APIs do not require management API keys to use. Non-administrator users of an Algorithmia platform will be able to use their simple API keys to interact with algorithms they have permissions to work with. The use of Management API keys will not be supported by these algorithm APIs.
You can find the details for each of these APIs available in the OpenAPI spec.
The OpenAPI spec allows developers to generate language-specific clients through code generators, test APIs, interact with APIs more easily. It can even generate a Postman collection that can be imported into your Postman workspace. You can find many of these tools on the OpenAPI website.
Update Algorithm Settings
Administrators and data scientists asked for a way to be able to update an algorithm’s settings without going through a full publish. For example, many users wanted to update their algorithm’s network access permissions or change the source code visibility after creating it.
Each of these settings and more is now available on the Manage tab of each algorithm. Most of the settings will apply immediately so data scientists can test private, unpublished versions.
However, to make settings changes available to developers consuming your algorithm, you’ll need to go through a full publish.
The two immediate settings that don’t require a publish are Source Visibility and License:
How to Update Algorithm Settings
During the publishing process, we’ll detect which settings will be changed for the next published version and inform the publisher the type of Semantic Versioning increase that will be required.
Note: All previous versions of a published algorithm are immutable and unable to be changed in order to help developers who are already using a specific version in their production apps.
New Data Storage Connectors
This release includes two new data storage connectors to allow data scientists and developers to leverage additional input and output locations for algorithms.
The data connector system allows an Algorithmia instance to securely handle brokering connections between an algorithm and a data source provided by the developers without needing to provide connection credentials.
Additionally, since it’s a single data API for the algorithm developer, access to an underlying data source is abstracted, allowing for algorithm developers to support many types of data sources without needing to implement provider-specific logic.
The two data connectors that are included in this release are:
Algorithmia .NET SDK Supports .NET Core and .NET Standard
Customers have been adopting and transitioning to .NET Core and have been asking us to transition from only supporting the .NET framework. We’re happy to now include support for the Algorithmia .NET Client in all of the frameworks supported by .NET Standard 2.0, which includes .NET Core. This means that many more application types can run across Windows, Linux, MacOS, and mobile, and other devices will be able to directly make calls to algorithms.
Supporting .NET Standard 2.0 provides support for the following frameworks:
- .NET Core
- ASP.NET Core
- .NET Framework (>4.6.1)
- Universal Windows Platform Apps
All of the Algorithmia Clients are open-sourced on GitHub. You can follow the .NET Client on the GitHub website.
To get going, you can install the updated NuGet package for the Algorithmia .NET Client and follow the walkthrough available in the Developer Center.
Flexibility to Enable User Work Limits and Queuing
Administrators need the flexibility to tune an algorithm’s usage of infrastructure while still running in production. Algorithmia has had the ability to provide “sticky slots” as one such flexibility example. This release introduces a new flexibility option to specify work limits for a particular user running production workloads.
This option will set limits to accept calls from a user but will begin queueing requests that are above the maximum active jobs limit until the maximum concurrent sessions are reached. The setting is available on the Advanced options page of the Admin Panel by clicking the “Update User’s Work Limits” button.
Administrators now have the flexibility to manage an algorithm’s use of their cluster. In one Algorithmia cluster, administrators were able to see a 30-percent decrease in monthly infrastructure costs by being able to finely tune production algorithms.
How to Update a User’s Work Limits
User to Update: Enter the username of the user you want to limit. Max Concurrent Sessions: Input a max number of concurrent requests a user can make at one time to ensure resources are not being over allocated. Max Active Jobs: Insert a max number of active jobs a user can have before requests begin queueing to ensure other users have resources available to them.