Deepspeed : AttributeError: 'DummyOptim' object has no attribute 'step'...

DeepSpeed Lightning refusing to parallelize layers even when setting to stage 3...

Problems when profiling LLM-training using "huggingface/accelerate" to Night system...