understand through listening

understand through listeningIt is an AI assistant launched by Aliyun, which is mainly used for processing and understanding audio and video content.

Product Overview

• Publishing BackgroundTongyi Hearing Wisdom was officially released on June 1, 2023 at the "Wisdom in Guangdong, Hong Kong and Macao" AliCloud Summit, which is another important product of AliCloud's big model layout.
• product positioning: AI assistant for work and study, designed to improve user efficiency in scenarios such as meetings, lectures, interviews, translations and more.

core functionality

1. real time transcription: The ability to quickly transcribe audio and video content into well-organized text shorthand.
2. Quick chapter overview: Distill the gist of multiple chapters and correspond directly to each node on the timeline.
3. Summary of statements: Distinguish between a number of different speakers at a conference and distill and summarize their points separately.
4. full abstract: Quickly summarize the core content of the video and shorten the long text to a synopsis.
5. One-click AI rewriting(New after upgrade): Automatically convert spoken content into written expressions.
6. Automatic generation of mind maps(New after upgrade): Automatically extract audio and video key points to generate mind maps.
7. Audio/video quiz assistant "Xiaowu"(New after upgrade): Support free Q&A within a single audio/video, also support answering questions across multiple audio/video recordings, and even support asking English videos directly in Chinese and giving answers directly in Chinese.

Technical characteristics

• semantic understanding: On the basis of transcribing sounds into words, Tongyi Listening and Understanding is able to understand the semantics and realize "listening" before "understanding".
• Efficient processingA 10-20 minute long audio/video can be converted into text at 10-100 times the acceleration ratio and understood in less than 1 minute.
• Multi-language support: Xiaowu Assistant supports Chinese directly ask English video, directly give the answer in Chinese, eliminating the translation step.

application scenario

• proceedings: Automatically differentiate between speakers and summarize ideas to enhance meeting efficiency.
• Learning Notes: Quickly generate lesson points and mind maps to aid learning.
• compile an interview: Transcribe and summarize interviews quickly for subsequent editing.
• translation assistance: Supports multi-language Q&A to aid translation efforts.

Usage

• Users can register for an account and open the Tongyi Hearing and Wisdom service through the AliCloud website.
• Newly activated service users can try the service for free for 90 days, with a daily free usage quota of 48 hours (transcription duration), and after the free quota is used up on the same day, it can only be reused after 24 hours.
• Users can upload audio and video files or use the microphone for real-time transcription.

future development

• Tongyi Hearing & Understanding's capabilities will continue to grow and delve into a range of verticals for customization to better meet user needs.
• Aliyun will be committed to play the role of Tongyi listening and understanding as an assistant to promote the rapid implementation of new technologies to empower thousands of industries.

Overall, Tongyi Hearing Wisdom is a powerful and easy-to-use AI assistant that can significantly improve users' efficiency in audio and video content processing. As technology continues to advance and applications continue to expand, Tongyi Listening is expected to play a greater role in the future.