Jay (@memmaptensor) / X

Jay

28 posts

Jay

@memmaptensor

Independent researcher

Bangkok

Joined December 2018

Following

628

Followers

Pinned
Jay
@memmaptensor
Jun 1
I spent $30k and 3 months RL post-training an anime video model. This is only step 30 out of a planned 1000 step run. All samples are local text-to-video with no reference image/audio. Since it's based on LTX-2.3, each output takes under a minute on a single GPU. I'm 19 and a
00:00
24K
Jay
@memmaptensor
Jul 1, 2024
I'm working on an architecture that can animate anime characters pretty consistently. It's about 40% trained. I still don't know what to do with the model once it's done.
00:00
38K
Jay
@memmaptensor
Jul 3, 2024
i've just realized it's been training for 40 days on 4xA100 SXM. it's going to take a bit more, around two weeks. after that, i'll try to get it to run in real-time (it's actually viable with the architecture). i think the hardest part about this project is funding, not as in
00:00
32K
Jay
@memmaptensor
Jul 14, 2024
yo @EsotericCofe @anifusion_ai wanna join forces best manga tool + best character animation model = ???
00:00
26K
Jay
@memmaptensor
Jul 21, 2024
While trying to push the CFG scale up, I implemented some Explicit RK solvers for ComfyUI - 10 new adaptive step samplers - 8 unique fixed step samplers (excluding forward euler) - Best new sampler (perhaps) -> fe_ralston3 Check it out! github.com/wootwootwootwo…
6.5K
Jay
@memmaptensor
Jul 3, 2024
Replying to @memmaptensor
the input image was this, btw
2.3K
Jay
@memmaptensor
Jul 15, 2024
i'd like to continue working on anime animation tech. version 1 is designed to be distilled for realtime inference. version 2 won't be concerned with realtime inference and would probably be based on a flow-matching mmdit with more fine-grained control and even better quality.
1.1K
Jay
@memmaptensor
Jul 14, 2024
Replying to @EsotericCofe and @anifusion_ai
Not realtime yet (still in roadmap) but we could get this deployed on anifusion before. The pose sequence is derived from mocap data from XR Animator, but we can probably work out a custom pipeline.
1.4K
Jay
@memmaptensor
Jul 24, 2024
Last major update! • Added solver settings for adaptive_scipy • Adaptive solvers now show the number of steps taken • Accurate 𝜎 timestep info is now displayed Check out the most comprehensive fixed and adaptive higher-order samplers on ComfyUI! github.com/wootwootwootwo…
2.6K
Jay
@memmaptensor
Jul 17, 2024
diffusion models are definitely still not dead. sure, optimal transport conditional flow matching is provably better, but so much of the community was already built on discrete time diffusion. and with kolors out (an SDXL model trained with DDPM formulation and eps-pred
1.4K
Jay
@memmaptensor
Jul 17, 2024
i have an idea for a slightly modified class of SDXL models that would mostly be compatible with existing finetunes and loras it's been proven to work well on SD1.5 with good results definitely next on my bucket list will post updates and releases soon, hopefully, if it works
953
Jay
@memmaptensor
Jul 4, 2024
my linkedin got restricted (for, using an anime pfp, presumably) so this is honestly great
1.7K
Jay
@memmaptensor
Jul 15, 2024
training is done in 2 days!!! as for inference compute requirements: it's basically the same burden as animatediff will probably work on realtime inference next month after figuring out life. realtime txt/vid2vid on distilled models is already empirically shown to be possible.
996
Jay
@memmaptensor
Jul 22, 2024
Refactored and fixed some bugs with the progress bar! Also wrapped the solvers from scipy.integrate If you count the a-methods as 2 (since they work with both the adaptive_pid and fixed_scheduled controllers), then this node has (excluding forward euler) 31 new samplers! I
698