Introduction

The EasyControl workflow is feature-rich and powerful, bringing many conveniences and possibilities to image creation. It has four practical functions: First, it supports ControlNet's Canny, OpenPose, and Depth, and can also be used with style LoRA, allowing you to easily achieve a variety of image effect control; second, it can generate unique Ghibli-style pictures to create a dreamy artistic atmosphere; third, by uploading object pictures and scene pictures, the scene pictures are masked to ensure that the objects remain consistent when they are migrated to the mask, and accurate image fusion is achieved; fourth, after directly uploading the object picture, the prompt words can be used to maintain the consistency of the object while generating pictures of the object in different scenes. Whether you are pursuing creative expression or performing fine processing of images, this workflow can meet your needs and help you efficiently produce high-quality image works.

Lightweight Condition Injection LoRA Module (CIL):

• Isolated Design: The core idea of EasyControl is "isolation." It avoids modifying the main DiT backbone that processes text and noise. Instead, it introduces an independent "Condition Branch" for the conditional signal (e.g., Canny edge map, face image).
• Targeted LoRA: It does not use traditional LoRA, which acts on the model's backbone for fine-tuning. Instead, LoRA (Low-Rank Adaptation) is applied exclusively within this new condition branch, efficiently learning how to encode and align the conditional information. The weights of the original text and noise branches remain completely frozen.
• Advantages: This design offers multiple benefits:
• Lightweight: Each condition control module has a very small parameter count (around 15M in the paper), far less than ControlNet.
• Plug-and-Play: Since the backbone network is untouched, EasyControl modules can be easily loaded like plugins and coexist harmoniously with various customized base models or style LoRAs, minimizing conflicts.
• Zero-Shot Multi-Condition: Most impressively, even though each condition module is trained independently, the framework supports combining multiple different types of conditions (e.g., pose + face) in a zero-shot manner for complex control, achieving stable results. This is ensured by the subsequent Causal Mutual Attention mechanism.

https://github.com/jax-explorer/ComfyUI-easycontrol?tab=readme-ov-file

https://huggingface.co/Xiaojiu-Z/EasyControl

Recommended machine：Ultra Pro

Workflow Overview

Part 1 : Get hf_token

1.Click on the user's avatar.

2.Click the Access Tokens button.

3.Click the Create new token button.

4.Click the Read button.

5.Enter the name of the token in the input box.

6.Click the Create token button.

7.After clicking the Create token button, the hf_token will pop up. Copy and keep the token. The token will be used in the workflow.

Part 2 : controlnet

Step 1 : Select ‘controlnet’ Workflow

Step 2 : Enter token and upload image

1.Enter the hf_token created in Part 1.

2.If you enable the load_8bit function, you can use less graphics card memory to generate videos, but it will increase the time it takes to generate images.

3.Upload pictures.

Step 3 : Select Model

1.The model in the Load EasyControl Sytle Lora node is a flux lora model, not an EasyControl model.

2.Select the controlnet type, such as openpose, depth, canny.Need to be the same as the controlnet type in the AIO Aux Preprocessor node.

3.Select the controlnet type, such as openpose, depth, canny.Need to be the same as the controlnet type in the Load EasyControl Lora node.

Step 4 : Input prompt word and input image resolution

1.Input prompt word.

2.Modify the image resolution by adjusting the 'height' and 'width' parameters.

Part 3 : Flux EasyControl Migrate any subjects

Step 1 : Select ‘Flux EasyControl Migrate any subjects’ Workflow

Step 2 : Enter token and upload image

1.Enter the hf_token created in Part 1.If you enable the load_8bit function, you can use less graphics card memory to generate videos, but it will increase the time it takes to generate images.

2.Upload scene image.Right click on the node, select the 'Open In MaskEditor' button, enter the painting page, and paint where you want to generate the object.

3.Upload item pictures.

Step 3 : Input prompt word and select model

1.Input prompt word.

2.Select EasyContol Lora model.

Step 4 : Modify image resolution

Modify the image resolution by adjusting the 'height' and 'width' parameters.

Part 4 : Ghibli

Step 1 :  Select ‘Ghibli’ Workflow

Step 2 : Parameter Adjustment

1.Enter the hf_token created in Part 1.If you enable the load_8bit function, you can use less graphics card memory to generate videos, but it will increase the time it takes to generate images.

2.Upload image.

3..Input prompt word.

4.Select EasyContol Lora model.

5.Modify the image resolution by adjusting the 'height' and 'width' parameters.

Part 5 : subject

Step 1 : Select ‘subject’ Workflow

Step 2 : Parameter Adjustment

1.Enter the hf_token created in Part 1.If you enable the load_8bit function, you can use less graphics card memory to generate videos, but it will increase the time it takes to generate images.

2.Upload image.

3..Input prompt word.

4.Select EasyContol Lora model.

5.Modify the image resolution by adjusting the 'height' and 'width' parameters.