Recently we’ve seen a large growth in variations of the Transformer model (Efficient Transformers: A Survey). However, leveraging improved variations requires custom complicated implementations hidden in a variety of dense libraries. …

SparseML brings GPU inference speeds to the CPU. This means substantial cost-saving, efficiency, and more options when it comes to deployability. …

Torch ORT uses the ONNX Runtime to improve training and inference times for PyTorch models. — With Lightning Flash, all it takes is enable_ort=Trueto use Torch ORT when training Transformer based models, giving you the power to use all features Lightning provides, such as Callbacks, Logging, Mixed Precision, and Distributed Training with support for Advanced Distributed Plugins.

As a result of our recent Lightning Flash Taskathon, we introduced a new fine-tuning task backed by HuggingFace Wav2Vec, powered by PyTorch Lightning. Wav2Vec 2.0 is a popular semi-supervised audio model that has shown impressive results when fine-tuned to downstream tasks, such as Speech Recognition. Wav2Vec has achieved State-of-the-Art Word…

Lightning 1.1 reveals Sharded Training — train deep learning models on multiple GPUs saving over 50% on memory, with no performance loss or code change required! — In a recent collaboration with Facebook AI’s FairScale team and PyTorch Lightning, we’re bringing you 50% memory reduction across all your models. Our goal at PyTorch Lightning is to make recent advancements in the field accessible to all researchers, especially when it comes to performance optimizations. …

Train state-of-the-art speech recognition, NLP and TTS models at scale with NeMo and Lightning — NeMo (Neural Modules) is a powerful framework from NVIDIA, built for easy training, building and manipulating of state-of-the-art conversational AI models. NeMo models can be trained on multi-GPU and multi-node, with or without Mixed Precision, in just 3 lines of code. …

Reduce cost and horizontally scale deepspeech.pytorch using TorchElastic with Kubernetes. End-to-End Speech To Text Models Using Deepspeech.pytorch Deepspeech.pytorch provides training, evaluation and inference of End-to-End (E2E) speech to text models, in particular the highly popularised DeepSpeech2 architecture. Deepspeech.pytorch was developed to provide users the flexibility and simplicity to scale, train and deploy their own speech recognition models…

Allen Institute as a part of their open research efforts released a data dump of scholarly articles as an initiative to aid efforts in tackling COVID-19. This dataset contains 51,000 articles as of the date this article being written and is increasing in size. When searching the data, key word…

Over the past few years at Digital Reasoning we have been developing audio analytics software to be highly effective at processing the noisy, domain-specific voice data that we typically encounter within the trading operations of major banks. Within the Audio Research Team, rapid research cycles drive continual refinements to our…