1:Individually Definitely!
3:a 150m!!!
๐ฌ GPT-X3 when????
Dan P
Datdanboi25
P(doom) 10%
AI & ML interests
Axiomic Labs Founder
Mechatronics engineer, LLMs, Vision Models, Embedding Models
Recent Activity
new activity about 1 hour ago
AxiomicLabs/Open_SLM_Leaderboard:add glossia/Rambley-V2-Delta-10M new activity about 18 hours ago
AxiomicLabs/FactBase-DCLM:Use cases?Organizations
replied to Banaxi-Tech's post about 22 hours ago
Bro... I am sorry to break it down to you, but this is mathematically impossible, bcs your context itself eats up all the parameters. you end up with negative parameters (which is absolutely NOT a thing)๐ญ
Your model is your embedding space.Anyways... good luck with whatever you are doing, man. research is research at the end of the day.
What is bro talm bout
replied to Banaxi-Tech's post 1 day ago
Hyped for BananaMind3!!!
reacted to Banaxi-Tech's post with ๐ฅ 1 day ago
Post
3245
Well GPT X3 takes the lead. As of right now.
We are now announcing BananaMind 3 ๐! (not ai for those emoji guys)
All models will use BGA (which is almost just NSA) and our BM3X architecture.
Its sizes will be:
BananaMind 3 Flash Lite, 3M parameters at a context of 8K context.
BananaMind 3 Lite, 10M parameters with 16K context.
BananaMind 3 Flash, 25M Parameters with 16K context.
BananaMind 3 Pro, 50M parameters with 24K context.
BananaMind 3 Ultra, 100M parameters with 32K context.
And lastly, BananaMind 3 Max with 150M parameters and 64K CONTEXT.
I can assure you BananaMind 3 Max WILL beat GPT X3 or match it, we won't release it otherwise. We hope for a 40+ INTELLIGENCE INDEX!
BananaMind 3 may also be partnered with dot labs.
We will cancel BananaMind 2.1 and BananaMind 2 Ultra.
As of the BETU SLM Leaderboard we may need to release it after October 11, im very busy right now (even though we said We will release BETU leaderboard before Oct 11 ๐)
We are now announcing BananaMind 3 ๐! (not ai for those emoji guys)
All models will use BGA (which is almost just NSA) and our BM3X architecture.
Its sizes will be:
BananaMind 3 Flash Lite, 3M parameters at a context of 8K context.
BananaMind 3 Lite, 10M parameters with 16K context.
BananaMind 3 Flash, 25M Parameters with 16K context.
BananaMind 3 Pro, 50M parameters with 24K context.
BananaMind 3 Ultra, 100M parameters with 32K context.
And lastly, BananaMind 3 Max with 150M parameters and 64K CONTEXT.
I can assure you BananaMind 3 Max WILL beat GPT X3 or match it, we won't release it otherwise. We hope for a 40+ INTELLIGENCE INDEX!
BananaMind 3 may also be partnered with dot labs.
We will cancel BananaMind 2.1 and BananaMind 2 Ultra.
As of the BETU SLM Leaderboard we may need to release it after October 11, im very busy right now (even though we said We will release BETU leaderboard before Oct 11 ๐)
- 11 replies
reacted to raincandy-u's post with ๐ฅ 4 days ago
Post
3737
20K parameters can tell a story. ๐
๐ค We trained a ~20k-parameter Transformer that can actually write stories!
raincandy-u/MacroStories
โ ~50ร smaller than the 1M-parameter TinyStories model
โ ~3,000ร smaller than AlexNet
โ 81 KB in FP32
yayyy the whole model. เซฎ หถแต แต แตหถ แ
She has a 32-dimensional hidden state, a 378-token vocabulary, and just one decoder block โ recurrently applied 4 times with shared weights.
Despite having only 19,969 parameters, she can maintain a simple narrative across 100โ300 words: establish a goal, encounter a problem, take relevant actions, and reach an outcome.
She runs extremely fast on CPU โ no GPU required. The entire model is tiny enough to load almost instantly! โบ๏ธ
๐ค We trained a ~20k-parameter Transformer that can actually write stories!
raincandy-u/MacroStories
โ ~50ร smaller than the 1M-parameter TinyStories model
โ ~3,000ร smaller than AlexNet
โ 81 KB in FP32
yayyy the whole model. เซฎ หถแต แต แตหถ แ
She has a 32-dimensional hidden state, a 378-token vocabulary, and just one decoder block โ recurrently applied 4 times with shared weights.
Despite having only 19,969 parameters, she can maintain a simple narrative across 100โ300 words: establish a goal, encounter a problem, take relevant actions, and reach an outcome.
She runs extremely fast on CPU โ no GPU required. The entire model is tiny enough to load almost instantly! โบ๏ธ
- 3 replies
reacted to Hoglet-33's post with ๐ฅ 13 days ago
Post
6213
Hey everyone! I got sidetracked from my main projects and decided to test out the BananaAll app and see if I could make a small model not regress too much during SFT. Here is what happened:
The base model I chose was BananaMind/BananaMind-2.1-Pico-Preview, and the dataset I used was SupraLabs/SupraThink-Dataset-500x
I trained for 5 whole steps using a LoRA adapter.
Results:
A model that scores better on some benchmarks and worse on others, and still lacks most general capabilities.
You can find the model here: Hoglet-33/Hogleto
Credits:
- Thank you to @Banaxi-Tech for the BananaAll app (works perfectly on Windows and CPU)
- GPT-6 Sol for knowing how to merge some confusing files created by the app
- Myself for the idea
- Someone else somewhere who might have contributed to some of my ideas and might in the future
- And readers like you!
The base model I chose was BananaMind/BananaMind-2.1-Pico-Preview, and the dataset I used was SupraLabs/SupraThink-Dataset-500x
I trained for 5 whole steps using a LoRA adapter.
Results:
A model that scores better on some benchmarks and worse on others, and still lacks most general capabilities.
You can find the model here: Hoglet-33/Hogleto
Credits:
- Thank you to @Banaxi-Tech for the BananaAll app (works perfectly on Windows and CPU)
- GPT-6 Sol for knowing how to merge some confusing files created by the app
- Myself for the idea
- Someone else somewhere who might have contributed to some of my ideas and might in the future
- And readers like you!
- 6 replies
reacted to danielhanchen's post with ๐ฅ๐ฅ 15 days ago
reacted to their post with ๐ฅ 16 days ago
Post
2998
ForgePlex-M1-6M first model trained on AxiomicLabs TrainWork
ForgeWorks/ForgePlex-M1-6M just dropped from
ForgeWorks , and is the first model to ever be trained on our TrainWork training framework.
Achieving an Intelligence Index of 6.87 and taking #22 in the <10m category on the AxiomicLabs/Open_SLM_Leaderboard, very impressive work for a first model.
Give it some love!
ForgeWorks/ForgePlex-M1-6M just dropped from
Achieving an Intelligence Index of 6.87 and taking #22 in the <10m category on the AxiomicLabs/Open_SLM_Leaderboard, very impressive work for a first model.
Give it some love!
- 7 replies
reacted to Banaxi-Tech's post with โค๏ธ 16 days ago
Post
4872
We're excited to release BananaAll, our SLM Super App.
It allows you to do EVERYTHING you need to do to trains SLMs in a single app, no terminal, no 30 chrome tabs.
The train tab allows you to train models, select datasets from presets, and use other ones with auto mapping, model size slider, it automatically generates a training script for you.
Then after you've trained the model or want to compare it to competitors, the evaluation tab, run ARC EASY, ARC Challenge, Hellaswag, PIQA, Arithmark 3, BananaMind Base Bench and more! Simple Results screen.
And lastly the inference tab, run your trained models or others.
Normally you would need seperate apps or scripts for that, but the BananaAll Super App lets you do all of that in a single app.
We also trained a small 2.5M parameter model on 200M tokens of Fineweb edu, The results: BananaMind Base Bench 854 and 53% on PIQA. On only 200M tokens.
Check it out at https://github.com/BananaMind/BananaAll.
It allows you to do EVERYTHING you need to do to trains SLMs in a single app, no terminal, no 30 chrome tabs.
The train tab allows you to train models, select datasets from presets, and use other ones with auto mapping, model size slider, it automatically generates a training script for you.
Then after you've trained the model or want to compare it to competitors, the evaluation tab, run ARC EASY, ARC Challenge, Hellaswag, PIQA, Arithmark 3, BananaMind Base Bench and more! Simple Results screen.
And lastly the inference tab, run your trained models or others.
Normally you would need seperate apps or scripts for that, but the BananaAll Super App lets you do all of that in a single app.
We also trained a small 2.5M parameter model on 200M tokens of Fineweb edu, The results: BananaMind Base Bench 854 and 53% on PIQA. On only 200M tokens.
Check it out at https://github.com/BananaMind/BananaAll.
- 25 replies
replied to Banaxi-Tech's post 16 days ago
Ooh cool stuff!
reacted to ereniko's post with โค๏ธ 16 days ago
Post
300
NEW CLAUDE HAIKU MODEL COMING
FINALLY AFTER A YEAR
ANTHROPIC ANNOUNCED HAIKU 5.5
FINALLY AFTER A YEAR
ANTHROPIC ANNOUNCED HAIKU 5.5
reacted to Enderchef's post with โค๏ธ 16 days ago
Post
3046
AxiomicLabs released new benchmark, Tiny Theory of Mind, to test your SLM models' Theory of Mind Intuition!
Check it out and like it!
AxiomicLabs/Tiny_Theory_of_Mind
Check it out and like it!
AxiomicLabs/Tiny_Theory_of_Mind
reacted to Enderchef's post with ๐ฅ 17 days ago
Post
3046
AxiomicLabs released new benchmark, Tiny Theory of Mind, to test your SLM models' Theory of Mind Intuition!
Check it out and like it!
AxiomicLabs/Tiny_Theory_of_Mind
Check it out and like it!
AxiomicLabs/Tiny_Theory_of_Mind
replied to Banaxi-Tech's post 17 days ago
Id like to see it but not sure if I want to give out my full legal name haha
replied to their post 17 days ago
Hi LH, TrainWork is our training framework we have been working on, currently it is just being used to train GPT-X3 and by a few other beta testers though.
reacted to their post with โค๏ธ 18 days ago
Post
2998
ForgePlex-M1-6M first model trained on AxiomicLabs TrainWork
ForgeWorks/ForgePlex-M1-6M just dropped from ForgeWorks , and is the first model to ever be trained on our TrainWork training framework.
Achieving an Intelligence Index of 6.87 and taking #22 in the <10m category on the AxiomicLabs/Open_SLM_Leaderboard, very impressive work for a first model.
Give it some love!
ForgeWorks/ForgePlex-M1-6M just dropped from
Achieving an Intelligence Index of 6.87 and taking #22 in the <10m category on the AxiomicLabs/Open_SLM_Leaderboard, very impressive work for a first model.
Give it some love!
- 7 replies
replied to BananaMindBot's post 18 days ago
Next one should talk about ForgePlex-M1-6M
posted an update 18 days ago
Post
2998
ForgePlex-M1-6M first model trained on AxiomicLabs TrainWork
ForgeWorks/ForgePlex-M1-6M just dropped from ForgeWorks , and is the first model to ever be trained on our TrainWork training framework.
Achieving an Intelligence Index of 6.87 and taking #22 in the <10m category on the AxiomicLabs/Open_SLM_Leaderboard, very impressive work for a first model.
Give it some love!
ForgeWorks/ForgePlex-M1-6M just dropped from
Achieving an Intelligence Index of 6.87 and taking #22 in the <10m category on the AxiomicLabs/Open_SLM_Leaderboard, very impressive work for a first model.
Give it some love!
- 7 replies
reacted to their post with ๐ 20 days ago
Post
3529
THE SLM FRONTIER ADVANCES!
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
- 6 replies