ctranslate2 online with Winfy

We have hosted the application ctranslate2 in order to run this application in our online workstations with Wine or directly.

Quick description about ctranslate2:

CTranslate2 is a C++ and Python library for efficient inference with Transformer models. The project implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering, etc., to accelerate and reduce the memory usage of Transformer models on CPU and GPU. The execution is significantly faster and requires less resources than general-purpose deep learning frameworks on supported models and tasks thanks to many advanced optimizations: layer fusion, padding removal, batch reordering, in-place operations, caching mechanism, etc. The model serialization and computation support weights with reduced precision: 16-bit floating points (FP16), 16-bit integers (INT16), and 8-bit integers (INT8). The project supports x86-64 and AArch64/ARM64 processors and integrates multiple backends that are optimized for these platforms: Intel MKL, oneDNN, OpenBLAS, Ruy, and Apple Accelerate.

Features:

• Encoder-decoder models supported
• GPT-2, GPT-J, GPT-NeoX, OPT, BLOOM supported
• Automatic CPU detection and code dispatch
• Fast and efficient execution on CPU and GPU
• Quantization and reduced precision
• Multiple CPU architectures support
• Dynamic memory usage
• Parallel and asynchronous execution

Programming Language: C++.
Categories: