Four Machine Learning Tools for Creatives
Looking to experiment with machine learning but don’t know where to start? Here are my four favorites, across creative coding, imaging, text, and sound.
Runway ML, the all-in-one toolkit
What: Runway ML promises to “Make the Impossible.” Pretty lofty claim, right? The company has created an “app store” of machine learning tools for artists, designers, filmmakers and other digital creatives.
What to use it for: Runway ML is designed by creative coders, for creative coders. Public research models are usually implemented quickly into the library, giving creatives an easy way to integrate the latest ML tech in their practice.
Price-tag: There is a free plan, but you have to pay for training, hosted models, and using models in the cloud. The budget depends on your project.
Any cool projects? Runway began as Valenzuela’s thesis project at the Tisch School of the Arts at New York University, and he soon opened it up to artist peers. Well-regarded creators such as Robbie Barrat and Mario Klingemann have praised the tool. You can also check the monthly community reels (see video above) for a taste of what people are creating.
Neural filters on Photoshop
What: There’s no need to introduce Adobe Creative Cloud, the visual editor featuring Photoshop and Illustrator, used by 22 millions designers. In November 2020, Adobe released Neural filters, a library of filters that aims to reduce workflows to a few clicks by using machine learning powered by Adobe Sensei. The tool generates new contextual pixels that are not actually present in your original image so you can change coloring and even add a whole new range of emotions to a portrait. The extra pixels can also provide super-resolution to the image, by making it look sharper than it actually is.
What to use it for: For advanced retouches and experimentation. Fellow Mac users, this is actually the only way to run GANS.
Price-tag: Depending on the Adobe apps you want to use, from $9 to $50 per month.
Any cool projects? I couldn’t find any artwork admitting or claiming to use Neural Filters. But I am pretty sure some are.
OpenAI GPT-3
What: GPT-3 is a cutting-edge language model that uses machine learning to produce human-like text. It takes in a prompt and attempts to complete it.
What to use it for: GPT-3 (and its older brother, or sister, GPT-2) is loved by the AI Art community, and many projects, from novels to Reddit bots and Dungeons & Dragons games have been created. What I find particularly interesting about GPT-3 is that it allows creators to operate at the system level.
Pricing: There are 3 pricing tiers, based on how many tokens you use. Check out this blog post breaking down budgets for different projects.
Any cool projects? The Guardian featured an article called A robot wrote this entire article. Are you scared yet, human?, authored by GPT-3. You can tell from the title the objective was a tad sensationalist… Some criticized the article for its opacity, as the Guardian recognizes it tweaked the results created by GPT-3.
An artist produced an entire gallery show of artworks he created after prompts provided by GPT-3. I love the Marcel Duchamp-esque “A Short History of Plungers and Other Things That Go Plunge in the Night”
Flow Machines
What: Powered by Sony, Flow Machines is a research and development project that aims to expand the creativity of musicians and composers. Including it here is a bit of a cheat code, as it is not open-source, and it looks like you have to be a Sony collaborator to have access to the model.
What they use it for: In 2016, Sony unveiled Daddy’s Car, a song created by the system to sound like The Beatles. I’m guessing the technology could allow creating many other “sound-alike” pieces of music.
Any other cool projects in the machine learning & sound field?
Contemporary composers such as Holly Herdnon also use machine learning. The American-born, Berlin-based artist collaborated with an AI called “Spawn” she co-created, on her last album Proto. The album features “live training” sessions during which vocalists recruited by Herndon and collaborator Dryhurst would teach Spawn how to interpret sounds.
Ready to create?
