But higher LR makes the path more volatile/unstable ? @nroggendorff
Pierre Colle
piercus
AI & ML interests
HD AI
Recent Activity
Organizations
replied to
their
post
16 days ago
posted
an
update
16 days ago
Post
251
The Loss is a field with holes ๐ณ๏ธโณ๏ธ
๐ Batch Size is your wheel size.
๐๏ธ Learning Rate is your speed.
๐ Batch Size is your wheel size.
๐๏ธ Learning Rate is your speed.
-
- 3 replies
posted
an
update
3 months ago
Post
3966
Starts erasing! ๐ ๐ ๐
This is made with a one-step SD1.5 LBM [1]ย eraser !
Data is open. Data pipeline is open. Training code is open.
On our LBM fork : https://github.com/finegrain-ai/LBM
[1] LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
This is made with a one-step SD1.5 LBM [1]ย eraser !
Data is open. Data pipeline is open. Training code is open.
On our LBM fork : https://github.com/finegrain-ai/LBM
[1] LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
- 1 reply
posted
an
update
3 months ago
Post
406
๐ง Reproducing LBM-Eraserโฆ in the open [1] !
A major recent paper on erasing is OmniEraser [2].
They open-sourced an evaluation dataset [3] (and I'm using it for the evaluation of our LBM-Eraser ๐).
It's not a big dataset (70 samples), but it's good quality pairs, and that's what matters !
cc @BaiLing
[1] Finegrain LBM Fork : https://github.com/finegrain-ai/LBM
[2] OmniEraser: VDOR: A Video-based Dataset for Object Removal via Sequence Consistency (2501.07397)
[3] BaiLing/RemovalBench
[4] LBM paper: LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
A major recent paper on erasing is OmniEraser [2].
They open-sourced an evaluation dataset [3] (and I'm using it for the evaluation of our LBM-Eraser ๐).
It's not a big dataset (70 samples), but it's good quality pairs, and that's what matters !
cc @BaiLing
[1] Finegrain LBM Fork : https://github.com/finegrain-ai/LBM
[2] OmniEraser: VDOR: A Video-based Dataset for Object Removal via Sequence Consistency (2501.07397)
[3] BaiLing/RemovalBench
[4] LBM paper: LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
posted
an
update
3 months ago
Post
848
๐ง Reproducing LBM-Eraserโฆ in the open [1] !
Today we have trained a LBM [2] promptless inpainter using
We use a subset of 1.25M images with
2 takeaways :
๐ผ Inpainting is better compared to our RORD experiments [5]
๐ฆถ "4 steps" outperforms single-step
[1] Finegrain LBM Fork : https://github.com/finegrain-ai/LBM
[2] LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
[3] supermodelresearch/Re-LAION-Caption19M
[4] Resolution-robust Large Mask Inpainting with Fourier Convolutions (2109.07161)
[5] https://huggingface.co/posts/piercus/778833977889788
cc @supermodelresearch @presencesw
Today we have trained a LBM [2] promptless inpainter using
Re-LAION-Caption19M[3].We use a subset of 1.25M images with
aesthetic_score > 5.6 and pwatermark < 0.2 and LaMa [2] mask generation.2 takeaways :
๐ผ Inpainting is better compared to our RORD experiments [5]
๐ฆถ "4 steps" outperforms single-step
[1] Finegrain LBM Fork : https://github.com/finegrain-ai/LBM
[2] LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
[3] supermodelresearch/Re-LAION-Caption19M
[4] Resolution-robust Large Mask Inpainting with Fourier Convolutions (2109.07161)
[5] https://huggingface.co/posts/piercus/778833977889788
cc @supermodelresearch @presencesw
posted
an
update
3 months ago
Post
1859
๐ง Reproducing LBM-Eraserโฆ in progress! [1]
When repurposing a T2I model into a pure I2I model, thereโs always that orphaned text path โ what do we do with it? ๐ค
You can reuse it as learnable embeddings in multi-task setups [2], freeze an empty text prompt, distillate or prune the corresponding part.
In LBM, they take a clever route โ zeroing [3] and reshaping [4] the text-related cross-attentions into self-attentions.
This gives you fresh weights for I2I computation, nicely integrated into your SD architecture.
๐ References
[1] Our LBM Fork: https://github.com/finegrain-ai/LBM
[2] OmniPaint: OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting (2503.08677)
[3] LBM Zeroing: https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L147-L148
[4] LBM Reshaping: https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L100
When repurposing a T2I model into a pure I2I model, thereโs always that orphaned text path โ what do we do with it? ๐ค
You can reuse it as learnable embeddings in multi-task setups [2], freeze an empty text prompt, distillate or prune the corresponding part.
In LBM, they take a clever route โ zeroing [3] and reshaping [4] the text-related cross-attentions into self-attentions.
This gives you fresh weights for I2I computation, nicely integrated into your SD architecture.
๐ References
[1] Our LBM Fork: https://github.com/finegrain-ai/LBM
[2] OmniPaint: OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting (2503.08677)
[3] LBM Zeroing: https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L147-L148
[4] LBM Reshaping: https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L100
- 2 replies
posted
an
update
4 months ago
Post
3147
We have trained a LBM-Eraser with RORD-Dataset in the open ๐ฅ
๐ 1-step only inference, no distillation
๐ชถ Light backbone :SD1.5
๐ง Light training : converge in 6k steps
Now let's improve this, especially the inpainting capabilities. Stay tuned for more :-)
LBM paper : LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
Our LBM fork : https://github.com/finegrain-ai/LBM
๐ 1-step only inference, no distillation
๐ชถ Light backbone :SD1.5
๐ง Light training : converge in 6k steps
Now let's improve this, especially the inpainting capabilities. Stay tuned for more :-)
LBM paper : LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
Our LBM fork : https://github.com/finegrain-ai/LBM
posted
an
update
4 months ago
Post
218
In LBM paper, the noise and the conditioning image are merged into a single composite image.
Unlike other inpainting methods (which typically grey-mask the missing area), LBM replaces the masked region with uniformly sampled random pixels.
Intuitively, since LBM is trained from a text-to-image (T2I) model, those random pixels act as a strong signal to the pretrained model โ essentially saying: โThis is where you can do your generative magic.โ
LBM Paper: LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
Our fork (work in progress): https://github.com/finegrain-ai/LBM
Unlike other inpainting methods (which typically grey-mask the missing area), LBM replaces the masked region with uniformly sampled random pixels.
Intuitively, since LBM is trained from a text-to-image (T2I) model, those random pixels act as a strong signal to the pretrained model โ essentially saying: โThis is where you can do your generative magic.โ
LBM Paper: LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
Our fork (work in progress): https://github.com/finegrain-ai/LBM
replied to
their
post
4 months ago
We are starting from the "fine-mask" / "Ours (scratch)" path.
Table below from LBM paper : https://huggingface.co/papers/2503.07535
posted
an
update
4 months ago
Post
164
LBM-Eraser reproduction on going ...
We are starting with RORD Dataset-only !
Our RORD -> webdataset extraction script : https://github.com/finegrain-ai/LBM/pull/4
RORD paper : https://bmvc2022.mpi-inf.mpg.de/542/
Thanks @presencesw for presencesw/RORD
We are starting with RORD Dataset-only !
Our RORD -> webdataset extraction script : https://github.com/finegrain-ai/LBM/pull/4
RORD paper : https://bmvc2022.mpi-inf.mpg.de/542/
Thanks @presencesw for presencesw/RORD
-
- 1 reply
posted
an
update
4 months ago
Post
2933
We've just forked LBM to reproduce the LBM eraser results
Our fork : https://github.com/finegrain-ai/LBM
LBM paper: LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
LBM relighting demo : jasperai/LBM_relighting
Our fork : https://github.com/finegrain-ai/LBM
LBM paper: LBM: Latent Bridge Matching for Fast Image-to-Image Translation (2503.07535)
LBM relighting demo : jasperai/LBM_relighting
- 2 replies
reacted to
1aurent's
post with ๐ฅ
over 1 year ago
Post
4479
Hey everyone ๐ค!
Check out this awesome new model for object segmentation!
finegrain/finegrain-object-cutter.
We (finegrain) have trained this new model in partnership with Nfinite and some of their synthetic data, the resulting model is incredibly accurate ๐.
Itโs all open source under the MIT license ( finegrain/finegrain-box-segmenter), complete with a test set tailored for e-commerce ( finegrain/finegrain-product-masks-lite). Have fun experimenting with it!
Check out this awesome new model for object segmentation!
finegrain/finegrain-object-cutter.
We (finegrain) have trained this new model in partnership with Nfinite and some of their synthetic data, the resulting model is incredibly accurate ๐.
Itโs all open source under the MIT license ( finegrain/finegrain-box-segmenter), complete with a test set tailored for e-commerce ( finegrain/finegrain-product-masks-lite). Have fun experimenting with it!
reacted to
1aurent's
post with ๐ฅ
over 1 year ago
Post
2720
Hey everyone ๐ค!
Check out this cool little reproduction of the Clarity Upscaler (https://github.com/philz1337x/clarity-upscaler) using refiners (https://github.com/finegrain-ai/refiners): https://huggingface.co/spaces/finegrain/enhancer
Check out this cool little reproduction of the Clarity Upscaler (https://github.com/philz1337x/clarity-upscaler) using refiners (https://github.com/finegrain-ai/refiners): https://huggingface.co/spaces/finegrain/enhancer
- 3 replies