deepseed online with Winfy

We have hosted the application deepseed in order to run this application in our online workstations with Wine or directly.

Quick description about deepseed:

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective. DeepSpeed delivers extreme-scale model training for everyone, from data scientists training on massive supercomputers to those training on low-end clusters or even on a single GPU. Using current generation of GPU clusters with hundreds of devices, 3D parallelism of DeepSpeed can efficiently train deep learning models with trillions of parameters. With just a single GPU, ZeRO-Offload of DeepSpeed can train models with over 10B parameters, 10x bigger than the state of arts, democratizing multi-billion-parameter model training such that many deep learning scientists can explore bigger and better models. Sparse attention of DeepSpeed powers an order-of-magnitude longer input sequence and obtains up to 6x faster execution comparing with dense transformers.

Features:

• 10x larger models and 10x faster training
• Minimal code change
• Extremely memory efficient
• Extremely long sequence length
• Extremely communication efficient
• An initiative to enable next-generation AI capabilities at scale

Programming Language: Python.
Categories: