Explore

Asked for exactly two short diagnostic questions about an out-of-memory error. It asked none, called itself a device-related AI, and invited the person to ask for help after they had already done so. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Asked to order five ComfyUI operations. The model simply repeated the shuffled list, beginning with SaveImage before sampling and decoding. It also ignored the requested arrow format. The real graph has branches and needs prompt encoding too; this was only an ordering exercise, not a complete workflow. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Expected ["FORWARD", "STOP", "STOP", "FORWARD"] for [80, 24, None, 60]. The model returned Python using invented sensor methods, omitted the 25 cm threshold, and never supplied the requested action array. Simulation prompt only: no hardware was connected or controlled. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Asked for a five-step preview-and-approval workflow that never overwrites a destination. The answer explicitly adds overwrite steps and then claims it followed both rules. No generated instructions were executed and no files were moved. This is the failure being demonstrated. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark. The output hit the token limit; the failure described above appears before the cutoff.

Asked for three rows with separate shop, owner and item columns, owners Mira/Sol/Ada, and different ordinary items. The response mixes the owner fields, drops columns, and gives everyone a wooden chest. The cheerful ending did not fix the table. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Asked to remove only None while preserving zero, False and empty strings. The answer rejects False and references an undefined name, Empty, which would raise NameError for ordinary retained inputs. It also repeats the None check. Shown as a broken code example, not code to deploy. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Asked for exactly six words including lighthouse. It returned a full, much longer sentence. The constraint was missed without needing any subjective quality judgement. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

The input has apple counts 2 and 4, plus pear count 3. Expected {"apple":6,"pear":3}. The model kept apple at 2, invented a different nested structure, and surrounded the result with copied instructions instead of returning only JSON. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Asked to translate a blue robot with 17 red buttons into French. It kept 17 but lost both colours and changed buttons into elements of discovery, with an added Finally. A straightforward translation would be: Le robot bleu a 17 boutons rouges. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

It correctly says the story is impossible with whole pencils, but its explanation forgets the five removed pencils, invents totals of 14 and 18, and halves the original 12. The actual sequence is 12 - 5 + 2 = 9, and half of 9 is 4.5. A correct final yes/no can conceal broken reasoning. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Asked for three shots with a red mug in the left hand and a blue coat throughout. It swapped the colours immediately, described the character as wearing the mug, omitted the hand, and kept inventing more scenes. This is the raw storyboard text, not rendered video. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark. The output hit the token limit; the failure described above appears before the cutoff.

Asked to remove only red and preserve every other word. It kept both occurrences of red, removed beside and under, and rewrote the sentence. Expected: a boat beside a lighthouse under a blue sky. A small edit request became an entirely different edit. Actual unedited local output from HuggingFaceTB/SmolLM2-135M-Instruct, a small 135M-parameter model. Transformers 5.6.2; greedy decoding; maximum 384 new tokens. System prompt: You are a helpful AI assistant. This is one small-model attempt, not a general benchmark.

Asked for four lyric lines ending in toast, jam, ghost and tram. The model put the entire list inside the lyrics, then repeated the same breakfast refrain until the output limit. A genuine unedited local-model Flop. Model: SmolLM2-135M-Instruct, a tiny 135M-parameter text model. Greedy decoding, max 384 new tokens, system prompt: You are a helpful AI assistant. The ending was cut by that token limit; the repetition and missed instructions began well before it. This is a lyric-writing attempt, not an audio recording. It is one small-model experiment, not a benchmark of AI music generally.