AI-Driven Typeface Design

Overview

FontFacing is a desktop app designed to speed up workflows when designing typefaces and letterforms. At its core, the goal is to design the visual interface around a system which uses artificial intelligence and machine learning in a way which helps the user understand how the system is working under the hood and so that the user has input in the way the system is working, analyzing, and training.

Although the final deliverable isn’t meant to be a working AI system, the foundational research and overall concept has to point to a system with real-world feasibility and the visual design has to highlight the featured AI systems. This is not a platform where the AI is an afterthought, it has to be an integral part of the concept.

Details

Desktop App Concept 2023 14 Weeks Solo Project

How might we integrate artificial intelligence features into the user interface of a platform?

Start by researching the strengths and weaknesses of AI platforms so you can figure out where in the user flow it belongs.

30 different A's as input

36 different generated A's

Research & Background

I started this project by researching how training models are made and making a couple myself. By using the available tools at RunwayML I was able to train image generating models in order to see, firsthand, how an AI system analyzes the material its given and, thereafter, what it can produce.

After also doing some research into AI platforms working with vector graphics, I solidified the concept I had in mind: a design and editing platform working with SVGs, supported and driven by AI to make it faster and easier for designers to build out the entirety of a typeface.

description

A comparison of capital A's generated by ChatGPT and Adobe Illustrator's Beta AI

Left: a capital A made of XML/SVG markup generated by ChatGPT; right: a capital A generated in Adobe Illustrator via Adobe's beta AI generation.

Starting Insights

An A.I. system can’t become an expert on everything, but can be trained for a specific medium or on a specific topic, even combining areas.

Most of the thought for A.I. systems seems to be from the viewpoint of “what can A.I. do for you” rather than “what can you do with A.I.” and because of that, they usually default to pretty basic forms of input, like a text box, because the time and effort was put into training the system, not into how we interact with it.

Training comes in many forms but will produce inconsistent results by itself. For the AI system to work, it will also require reinforcement and supervision.

A.I. platforms can’t separate outliers unless specifically trained to do so. Therefore, every word/pixel/megabyte used in training needs to be carefully selected and vetted.

Because we don’t know what’s happening, what the system is doing, and how our input is being interpreted AI feels like magic when the system gives results.

Machine learning is exceedingly good at working with patterns but, due to the previous insights, everything you put into the training will be part of the resulting pattern.

A possible solution would be to give designers access to different kinds of input while showing what the AI does with it.

By showing a designer what the AI is doing/thinking/processing with their input, instead of just the end result, they can better incorporate the AI features into their workflow.

A low fidelity wireframe from the first round of ideating

Starting to Design

When I started to design, my original idea was a webapp that was much more aimed at typeface design using some basic text entry and drawing tools. The incorporation of AI was, at most, an additional feature or tool rather than the main driving force. I ended up pivoting to a desktop app since I’d never designed for one.

In the process I looked to familiar design platforms, the obvious ones like Adobe, and some type design platforms, like FontLab, for some inspiration.

description

A screenshot from a typeface designing software called FontLab Studio

Screenshot from FontLab

How I imagine the AI would analyze input to create a model for each new design file.

We need to refine the solution by further defining the workflow loop within the program.

Since AI is the driving force of the platform the designer should have access to it whenever they need or want, but there should also be a specific area or stage when the AI is always active and working.

An image showing the different possible workflow loops through each AI method, including file uploads, text prompts, and drawing tools. The workflows show that each method leads into the AI training model and then loops back to the start, showing that any input will affect the training model as you work.

Making Revisions

As I designed, I realized that the AI features, as I’d implemented them, were still a simple addition to the existing suite of drawing and editing tools. The first medium fidelity version of a real AI implementation was a simple modal which opened when you started a new typeface file, along with some context menu options.

I wanted the platform to feel like a fully invested piece of AI-driven software, so I had to revise the overall design so that the AI features were a constant with prominent calls to action. I moved the AI features to the menu bar, always there, to push them into a kind of “more important” state than the rest of the tool bar.

description

A medium fidelity wireframe showing a pop-up modal with tabs to choose the method by which to start working on a new design file

Above: the original concept of introducing the AI features occurred as a modal when starting a new file, after which they would be available in context menus. Below: the updated concept of giving the AI features a permanent home in the menu/mode bar.

A higher fidelity wireframe showing a toolbar in a desktop application which features permanent AI generation features

With the workflow and design resolved, expand the visual language of the platform.

Create a consistent user experience by creating a design system with defined color palette, shape language, and font choices.

Finishing Up

I think designers can easily get sucked into the common pitfall of thinking that their first idea is the best idea. However, when I look at this project, the concept may not have changed all that much (although it definitely got much more in-depth) but the interface changed drastically.

I'm glad that I made the decision to switch from a webapp to a desktop platform. In terms of the GUI there aren't that many differences in capability (just look at Figma, who manages to make both versions work, although their desktop app is essentially just the webapp hosted in a custom browser) but the switch to a desktop platform helped to further inform and specify the kind of designer, workflow, and AI possibilities that I should aim for.

description

A mockup of the starting screen of FontFacing on a Macbook.

More Work

A preview image showing an iPhone app mockup next to the coded website.

Next Project

Tatt.ooo

Research & Background

When I started this project, I had to create a training model to understand how AI analyzes the content that it's trained on in order to form the training model that will then be used.

At the time, I had hundreds of photos of beers that I've taken over a few years just sitting around waiting to be used for something, so I made the first training model using around 1,000 of these photos. Some contain things in the background, like a bar, living room, and even people, and some are on a white background. Some show one, two, or three cans or bottles in various sizes. Most of them show a glass that's been filled with beer, and some show multiple glasses that have been filled with beer.

4 images that were created from the previous images.

This training model was interesting. I didn't know what to expect. The resulting images that were generated from it are actually a pretty good showing, but the photos I used for the model weren't selected or taken with AI training in mind (they weren't vetted and I didn't take a look at any of them prior to building the model). I wanted to make another model where the images were very specifically created with the purpose of being put into a training model in order to see how that would affect the generated images compared to the first model. I gathered 30 beers (all 16-ounce cans) and took new photos with each can against a white backdrop, although I staggered the angles and composition of the photos. This training model was different, but provided much more insight as to how the system was analyzing the given photos.

For the hands-on research, I made one last model with very simple images, 30 different versions of a capital 'A', each one black on a white background. The output from this (seen to the right) may seem like a poor result but, to the contrary, I believe that it serves as concrete proof that I'm thinking about the concept correctly: an AI training model is perfectly capable of finding patterns, consistent details, and unique attributes and then recreating those aspects... as long as it's given an appropriate context.

I did additional research regarding AI systems working on vector images, as well as the current prevalence of AI in the creation, recreation, and/or expansion of typographic systems. There's currently a lot of work put into text (copy) generation, but not a lot of work put into typeface or letterform generation.

In particular, I was inspired by the work of Erik Bern, Jean Böhm, and Måns Grebäck. These 3 designers (writing about their experiments in 2016, 2020, and 2022, respectively) have all worked in the space of typeface generation. Erik Bern used 50,000 fonts to train a neural network; Jean Böhm experimented with a workflow of AI abilities to generate SVG paths and anchor points to create letterforms; Måns Grebäck experimented with machine learning in the use of Stable Diffusion to generate new or stylistically similar typefaces to those he provided. All of their work served as proof that my concept could work, I just had to design an interface for it.

Aggregated letter generations from Erik Bern.

Typeface inputs (either lowercase or uppercase) alongside the AI-generated accompaniment (lowercase used to generate uppercase and vice versa) from Måns Grebäck.

Extensive experiments using AI to generate svg paths for a lowercase letter 'a' from Jean Böhm.

Starting to Design

In the process I looked to familiar design platforms, the obvious ones like Adobe, and some type design platforms, like FontLab, for some inspiration.

The first ideation, as a webapp, was not focused enough on being an empowering tool. The idea at this stage was a very easy to use, approachable webapp that was designed to be as simple as possible for new users to start designing, but was not a thorough design tool for serious typeface designers. This first exploration also didn't use the AI system to the fullest; it uses AI behind the scenes and generates a typeface through a basic text prompt system, with some uber simplistic drawing tools that can be used afterwards. This was inspired largely by the popularity of systems like ChatGPT and MidJourney, which were exploding in popular discussion and social awareness at the time I was designing this exploration.

A low fidelity wireframe of the desktop app.

It was very obvious that this concept needed a lot more. I started to look at platforms like Adobe Illustrator, FontLab, and Figma to see how popular design applications approach interface design as well as putting some more thought into the minimum level of tools that would be necessary to make a platform like this work for serious designers. At this stage, Adobe was starting to make public their beta AI tools and context menus, so I added a floating context menu to my next iteration. FontLab and Figma also heavily (almost entirely) utilize sidebars to show properties and options, so I added a sidebar to the right side to explore the use and representation of the AI analysis within the application.

One iteration points to the workflow of typeface design and the fact that there are different stages. As far as typeface design goes, there's always a basic sketching phase to explore the visual language, there's a focused stage where you can really get lost in the weeds of making tiny adjustments to every single anchor point, and there's a letter-spacing phase at the end where you can fine tune the spacial relationships between each character. What an AI-driven workflow would add is a "live generation" mode to this. What this means is that the sketching phase (whether sketching in the application using drawing/vector tools or uploading photos of analog sketches or inputting a text prompt to generate some imagery as an aesthetic basis) can be used as the input for the training model of an AI. The idea here is that once you enter into the "live generation" mode, the platform will build out the entirety of the typeface (all remaining characters that haven't been sketched, smooth anchor points, etc.) and then whenever you make changes to a character the AI can continuously take those changes as additional input and refine the rest of the typeface.

Before getting into the weeds of the visual design language for the interface, I did some more exploration of how to better incorporate the AI into a driving force and permanent aspect of the platform. To make it front and center, I brought it into a modal which would open immediately when starting a new file to allow some text input to generate a starting point or a file upload to allow immediate image training. Alongside this, I also did a variety of quick wireframes experimenting with the placement of the toolbar and some other elements.

This proved to be the final version of this interface, showing the entire typeface on a single, large artboard. The next versions would be a drastic change after some serious introspection and revisiting my research.

Making Revisions

To start working on my visual design language and really define the appearance and experience of using the interface, I did some explorations of verbiage relative to line, shape, color, form, space, and texture. This was a long process because I was never quite satisfied with the design aspects; none of them seemed quite right. (More on this later.)

An exploratory sketching exercise to try to figure out some of the finalized visual design language.

I took a step back from my interface design to ponder. The original idea to have one larger working area and showing all of the characters within the typeface, was to point to a workflow where you can easily zoom in and out of your work to see your micro changes against the overarching macro design of the typeface. I realized, however, that working on typefaces is often spent on details, zoomed into the letterforms; I also noticed that after adding the sidebar windows to showcase the AI analysis and generation (overarching navigation view in the bottom right) that they served a similar purpose of making the macro visible to the designer.

Alongside the change to individual artboards for each letterform, I needed a way to navigate between these artboards. When looking at the interface of Figma (a central working area bookended by sidebars on the left and right, with layers on the left and properties on the right), I realized that an artboard for a single character would be compact enough that having a sidebar on the left of the window wouldn't inhibit the working area and would also give me a space to house the "layers" of the template, guidelines, shapes, etc.

A finalized version of the interface which features additional elements, like another sidebar.

Final Thoughts

Narrowing down and specifying the audience that I was designing for helped the design process a lot. My initial concept was a much more casual experience with more basic interactions. Specifying that my audience would be type and graphic designers and type foundries (people and companies who are intimately familiar with design and design programs) helped me to design with more intention.

With more intention in mind, I was also able to better apply what I'd learned about AI and machine learning systems. In early stages, my concept didn't really address some of my initial research insights, for instance: my observation regarding AI feeling like magic. Adding the analysis window to a side bar (and the ability to confirm and even edit some of the program's analysis) allows the user to feel like they are still in control of the machine learning system, rather than an observer.

An early version of the analysis sidebar.

The final version of the analysis sidebar.

Something else that I've noticed within this process is the difficulty of working with gray as a primary color of a brand or experience. Since low fidelity and medium fidelity wireframing can often use shades of gray, when gray colors carry over into high fidelity designs it can be off-putting for collaborators and stakeholders. (And every professor I've ever had has appreciated shades of gray in graphic design work but not in user interfaces.) However, when we look at the primary competitors (FontLab, Adobe, Figma, etc.) their interfaces are inundated with shades of gray; light gray, dark gray, charcoal, gray is everywhere. There's always a prevailing idea that you shouldn't try to emulate your competition (and that's absolutely true, to a certain extent) but if they all have a similar design ethos then it should be indicative that a particular design ethos prevails for a reason: an interface which you'll be glued to for hours at a time can be overwhelming and distracting when too much color is involved.

A color exploration of the final interface and prototype.

The high fidelity prototypes that I produced for the end of the class are visual designs that I was not entirely proud of. The color system doesn't quite work, and I decided to continue working on the designs afterward. The revised designs are what you see here.