Steal Like An Artist
AI - Driven Shortfilm
Generative Tools with focus on ComfyUI Workflows
Introduction
My project aims to tell the story of a frustrated artist navigating the struggles of creativity. I initially explored AI tools like DALL·E and Ideogram for character creation and scene design. Still, while these platforms generated visually appealing images, they lacked consistency in the artist's appearance across poses and settings. This limitation led me to train a LoRA model in FluxGym to achieve a unified protagonist and adapt workflows in ComfyUI to blend LoRA-based character generation with fixed background images. Additional tools like Photoshop Beta AI for fine-tuning, Tensorpix for quality enhancement and a GPT-trained film director for scene planning further shaped the project. KlingAI became my main tool for animating still images into cohesive video scenes.
Development
Character Consistency & AI Assistance
- Developed/ Trained Dana's Film Assistant, an GPT-based AI inspired by Damien Chazelle, to guide scene structure, character arcs and visual depth
- Trained a LoRA model, Frankai, in FluxGym using a dataset of various diverse images to ensure consistent character design
- Integrated the model into ComfyUI workflows for uniformity across different poses and settings
Scene Creation & Refinement
- Used a dual-conditioning workflow in ComfyUI to blend LoRA-generated characters with fixed backgrounds
- Adjusted noise levels and seed values to balance character clarity and environmental accuracy
- Enhanced images in Photoshop Beta, modifying elements like hair, textures and props for artistic cohesion
Video Generation & Cinematic Structure
- Utilised KlingAI’s tools for photo-to-video transformations and interactive scene creation
- Managed AI-generated inconsistencies by refining character movements and adjusting animation pacing
- Implemented insights from movie director GPT to optimise shot compositions, pacing and transitions
For a more in-depth look at the development process and creative experimentation, click here to explore the full documentation.
Outcome
Reflections
Ethical Considerations:
A key ethical issue I encountered during this project was the prevalent presence of stereotypes and bias in AI-generated content. While the LoRA model I trained for my protagonist ensured consistency, its limitations highlighted broader issues in AI datasets. For example, the model was trained on a white male character, which influenced not only the protagonist’s appearance but also the surrounding cast. Men were generated far more often than women and when women did appear, they were often depicted with distorted features, leaving me questioning whether this bias stemmed from the LoRA model itself or deeper flaws in the AI training datasets.
This bias extended beyond character representation. While generating scenes in a European setting, prompts involving "neon lights at night" often included signs in Asian scripts, even though the environment had no cultural connection to Asia. Similarly, people of colour were rarely generated unless explicitly specified. At first, I was impressed when the AI included people of colour in the graffiti street scene, but later, upon deeper reflection, I realised this felt less like inclusivity and more like a reflection of bias in associating certain demographics with specific environments. Gender-based stereotypes also became apparent in wardrobe choices. When generating people in formal attire, men were consistently dressed in tuxedos or suits, while women appeared almost exclusively in red dresses, reinforcing outdated norms about gender roles.
These patterns emphasise the importance of holding AI developers accountable for ensuring diverse, unbiased datasets. Representation in AI-generated content isn't just about aesthetics, it has real-world implications for how technology keeps stereotypes alive and thereby impacts society’s perception of identity and inclusivity. This project highlighted the urgent need for ethical oversight in AI, especially when dealing with human representation.
Reflection: This project was a deep dive into how AI and machine learning can enhance and reshape the creative process. Tools like ComfyUI, Photoshop Beta and KlingAI allowed me to experiment with storytelling, visuals and animations in ways that would be impossible on a small budget and not in a professional setting. Photoshop Beta, with its hands-on approach, enabled precise edits, letting me fine-tune specific elements like colours or composition without regenerating entire scenes. ComfyUI’s node-based workflows allowed for a modular process where I could focus on balancing character consistency and background integration simultaneously.
Yet, the process wasn’t without challenges. AI-generated ideas for the film’s plot were often too generic, with predictable twists like “it was all a dream” or “he was dead the whole time.” This required human intervention to develop a more layered and original narrative. Visuals, too, demanded careful curation, out of several generated videos, only one or two were usable and even these required speed adjustments and editing to meet aesthetic goals.
The unpredictability of AI brought both frustrations and surprises. For instance, I imagined the protagonist sitting still in a café, but AI unexpectedly added him sipping coffee and sketching, a detail that elevated the scene’s depth. However, AI’s literal interpretation of prompts often posed issues, for example, asking for a "wobbly" motion caused objects to move unnaturally. I learned that negative prompts and weighting techniques became essential for guiding the AI effectively.
This project reaffirmed that while AI is a powerful tool for ideation and execution, the human touch remains essential. Maintaining cohesion, refining ideas and creating a narrative that resonates emotionally requires oversight that AI alone, at this stage, can’t provide. Despite its limitations, the process was highly rewarding, showcasing AI’s ability to accelerate workflows and inspire creativity. Ultimately, this project highlighted the evolving partnership between human creativity and technology in artistic fields.
This bias extended beyond character representation. While generating scenes in a European setting, prompts involving "neon lights at night" often included signs in Asian scripts, even though the environment had no cultural connection to Asia. Similarly, people of colour were rarely generated unless explicitly specified. At first, I was impressed when the AI included people of colour in the graffiti street scene, but later, upon deeper reflection, I realised this felt less like inclusivity and more like a reflection of bias in associating certain demographics with specific environments. Gender-based stereotypes also became apparent in wardrobe choices. When generating people in formal attire, men were consistently dressed in tuxedos or suits, while women appeared almost exclusively in red dresses, reinforcing outdated norms about gender roles.
These patterns emphasise the importance of holding AI developers accountable for ensuring diverse, unbiased datasets. Representation in AI-generated content isn't just about aesthetics, it has real-world implications for how technology keeps stereotypes alive and thereby impacts society’s perception of identity and inclusivity. This project highlighted the urgent need for ethical oversight in AI, especially when dealing with human representation.
Reflection: This project was a deep dive into how AI and machine learning can enhance and reshape the creative process. Tools like ComfyUI, Photoshop Beta and KlingAI allowed me to experiment with storytelling, visuals and animations in ways that would be impossible on a small budget and not in a professional setting. Photoshop Beta, with its hands-on approach, enabled precise edits, letting me fine-tune specific elements like colours or composition without regenerating entire scenes. ComfyUI’s node-based workflows allowed for a modular process where I could focus on balancing character consistency and background integration simultaneously.
Yet, the process wasn’t without challenges. AI-generated ideas for the film’s plot were often too generic, with predictable twists like “it was all a dream” or “he was dead the whole time.” This required human intervention to develop a more layered and original narrative. Visuals, too, demanded careful curation, out of several generated videos, only one or two were usable and even these required speed adjustments and editing to meet aesthetic goals.
The unpredictability of AI brought both frustrations and surprises. For instance, I imagined the protagonist sitting still in a café, but AI unexpectedly added him sipping coffee and sketching, a detail that elevated the scene’s depth. However, AI’s literal interpretation of prompts often posed issues, for example, asking for a "wobbly" motion caused objects to move unnaturally. I learned that negative prompts and weighting techniques became essential for guiding the AI effectively.
This project reaffirmed that while AI is a powerful tool for ideation and execution, the human touch remains essential. Maintaining cohesion, refining ideas and creating a narrative that resonates emotionally requires oversight that AI alone, at this stage, can’t provide. Despite its limitations, the process was highly rewarding, showcasing AI’s ability to accelerate workflows and inspire creativity. Ultimately, this project highlighted the evolving partnership between human creativity and technology in artistic fields.
Explore more of my Multimedia Work
