MindMap Gallery FFmpeg project practical column v6.0
Tencent Classroom Lingsheng Education FFmpeg Project Practical Column Course v6.0 The only audio and video tutorial that supports FFmpeg6.0 on the entire network Compatible with FFmpeg4.2 version Three major practical projects: 1. Player, 2. Recording push streaming, 3. Live streaming
Edited at 2023-05-18 14:54:25El cáncer de pulmón es un tumor maligno que se origina en la mucosa bronquial o las glándulas de los pulmones. Es uno de los tumores malignos con mayor morbilidad y mortalidad y mayor amenaza para la salud y la vida humana.
La diabetes es una enfermedad crónica con hiperglucemia como signo principal. Es causada principalmente por una disminución en la secreción de insulina causada por una disfunción de las células de los islotes pancreáticos, o porque el cuerpo es insensible a la acción de la insulina (es decir, resistencia a la insulina), o ambas cosas. la glucosa en la sangre es ineficaz para ser utilizada y almacenada.
El sistema digestivo es uno de los nueve sistemas principales del cuerpo humano y es el principal responsable de la ingesta, digestión, absorción y excreción de los alimentos. Consta de dos partes principales: el tracto digestivo y las glándulas digestivas.
El cáncer de pulmón es un tumor maligno que se origina en la mucosa bronquial o las glándulas de los pulmones. Es uno de los tumores malignos con mayor morbilidad y mortalidad y mayor amenaza para la salud y la vida humana.
La diabetes es una enfermedad crónica con hiperglucemia como signo principal. Es causada principalmente por una disminución en la secreción de insulina causada por una disfunción de las células de los islotes pancreáticos, o porque el cuerpo es insensible a la acción de la insulina (es decir, resistencia a la insulina), o ambas cosas. la glucosa en la sangre es ineficaz para ser utilizada y almacenada.
El sistema digestivo es uno de los nueve sistemas principales del cuerpo humano y es el principal responsable de la ingesta, digestión, absorción y excreción de los alimentos. Consta de dos partes principales: el tracto digestivo y las glándulas digestivas.
Tencent Classroom Lingsheng Education FFmpeg project practical column course v6.0 The only audio and video tutorial on the entire network that supports FFmpeg6.0 Compatible with FFmpeg4.2 version Three major practical projects: 1. Player, 2. Recording and pushing, 3. Live streaming
Audio and video basics
1.1 Audio basics
01. How to collect sound-Analog-to-digital conversion principle
02. Why high-quality audio sampling rate >=44.1Khz
03. What is PCM
04. How many bits are used to represent a sampling point?
05. Should the sampling value be represented by an integer or a floating point number?
06. The volume is closely related to the sampling value
07. How many sampling points are used as one frame of data?
08. How to arrange the sampling data of the left and right channels
09. What is PCM (Pulse Code Modulation)
10. Audio coding principles
1.2 Basic knowledge of video
01. RGB color principle
02. Why YUV format is needed
03. What is a pixel?
04. Resolution, frame rate, bit rate
05. Differences in YUV data storage formats
06. YUV memory alignment problem
07. Why does the screen display a green screen?
08. H264 encoding principle
09. Relationship between H264 I P B frames
1.3 Basic knowledge of demultiplexing
01. What is demultiplexing, such as MP4 format
02. Why different multiplexing formats MP4/FLV/TS are needed
03. Common multiplex formats MP4/FLV/TS
1.4 FFmpeg6.0 development environment construction
01. Three major platforms: Windows, Ubuntu and MAC
02. QT installation
03. FFmpeg command line environment
04. FFmpeg API environment
05. FFmpeg compilation
06. vs2019 installation (win platform)
1.5 Common tools for audio and video development
01. MediaInfo, analyze video files
02. VLC player, playback test
03. EasyICE, analyze TS stream
04. flvAnalyser, analyze FLV
05. mp4box, analyze mp4
06. audacity, analyze audio PCM
07. Elecard_streamEye, analysis H264
08. Hikvision YUVPlayer, analyzing YUV
FFmpeg cornerstone
2.1 FFmpeg command
01. Audio PCM/AAC file extraction
02. Video YUV/H264 file extraction
03. Demultiplexing and reuse
04. Audio and video recording
05. Video cropping and merging
06. Image/Video Conversion
07. Live push and pull streaming
08. Watermark/picture-in-picture/nine-square filter
Note: The purpose of mastering FFmpeg is: 1. Quickly grasp what FFmpeg can do; 2. Deepen your understanding of audio and video.
2.2 Practical implementation of SDL cross-platform multimedia development library
01. SDL environment construction
02. SDL event handling
03. SDL thread processing
04. Video YUV screen rendering
05. Audio PCM sound output
Note: SDL is compatible with the three major platforms of Win, Ubuntu and Mac. Mainly used for screen display and sound output of subsequent projects
2.3 Essential Lectures on the Cornerstone of FFmpeg
01. FFmpeg framework
02. FFmpeg memory reference counting model
03. Demultiplexing related AVFormat XXX, etc.
04. Codec related AVCodec XXX, etc.
05. Compressed data AVPacket
06. Uncompressed data AVFrame
07. FFmpeg object-oriented thinking
08. Zero copy of Packet/Frame data
Note: The purpose is to be familiar with FFmpeg’s common structures and function interfaces
2.4 FFmpeg audio and video demultiplexing and decoding
01. Demultiplexing process
02. Audio decoding process
03. Video decoding process
04. Analysis of FLV packaging format
05. MP4 packaging format analysis
06. What is the difference between FLV and MP4 seek?
07. Why FLV format can be used for live broadcast
08. Why MP4 cannot be used for live broadcast
09. Can MP4 be used for on-demand playback?
10. AAC ADTS Analysis
11. H264 NALU analysis
12. AVIO memory input mode
13. Audio resampling practice
14. Is the playback duration of the resampled data consistent?
15. How to represent PTS after resampling
16. YUV memory alignment problem after video decoding
17. PCM arrangement format problem after audio decoding
18. Hardware decoding dxva2/nvdec/cuvid/qsv
19. Transfer hardware gpu data to cpu
20. H265 decoding
Note: FFmpeg API learning: video demultiplexing -> Decode -> Encode -> Multiplex composite video
2.5 FFmpeg audio and video encoding multiplexing synthesized video
01. AAC audio encoding
02. H264 video encoding
03. PCM YUV multiplexing to synthesize MP4/FLV
04. H264 encoding principle
05. The difference between IDR frame and I frame
06. Dynamically modify the encoding bit rate
07. GOP interval reference value
08. The problem of out-of-synchronization of multiplexed and synthesized MP4 audio and video
09. Encoding and reusing timebase issues
10. MP4 synthesis cannot be played on IOS
11. How to express PTS after resampling
12. Video encoding YUV memory alignment problem
13. Hardware encoding dxva2/nvenc/cuvid/qsv
14. H265 encoding principle
15. H264 and H265 encoding conversion
FFmpeg advanced
3.1 FFmpeg filter
01. FFmpeg filter chain framework
02. Audio filter framework
03. Video filter framework
04. Multi-channel audio mixing amix
05. Video watermark
06. Video area cropping and flipping
07. Add logo to video
Note: Filter is widely used in the field of video editing.
3.2 ffplay player
01. Master the meaning of ffplay.c
02. ffplay framework analysis
03. Demultiplexing threads
04. Audio decoding thread
05. Video decoding thread
06. Sound output callback
07. Screen rendering time interval
08. Audio Resampling
09. Screen size format conversion
10. Differences in audio, video and external clock synchronization
11. Audio resampling compensation when using video as reference
12. The essence of muting and adjusting the volume
13. Audio and video packet queue size limit
14. Audio and video packet queue thread safety
15. Audio and video frame queue size limit
16. Audio and video frame queue thread safety
17. Pause and play implementation mechanism
18. Screen stuck problem caused by seek playback
19. seek playback data queue and synchronized clock processing
20. How to play frame by frame
21. Key points of player exit process
Note: ffplay.c is the source code of ffplay command, master ffplay It has the effect of getting twice the result with half the effort for us to develop our own players.
FFmpeg streaming media project practice
4.1 FFmpeg QT_0voice player project
01.Open source player analysis
02. Self-research framework analysis
03. Player module division
04. Demultiplexing module
05. Audio and video decoding
06. Player control
07. Audio and video synchronization
08. Software decoding Hardware decoding
09. Volume control
10. Variable speed playback
11. Drag to jump to play
12. Image brightness and saturation adjustment
13. Screen 4:3 16:9 switching
14. Code stream information analysis
15. Image rotation and flipping
16. Support screenshots
Project significance: Master the development of players, including player framework analysis, Module design, queue design, audio and video synchronization, speed change, drag and play and other core mechanisms.
4.2 FFmpeg RTMP recording and streaming_0voice recording and streaming project
01. Introduction to RTMP protocol
02. Recording and streaming framework analysis
03. Microphone and system sound capture
04. Sound capture delay analysis and solution
05. Camera and screen capture
06. Screen capture delay analysis and solution
07. Audio Resampling
08. Video scale
09. AAC encoder package
10. H264 encoder package
11. Optimization of audio coding parameters
12. Video encoding quality/CPU/bit rate/delay optimization
13. How to set timestamp when collecting audio and video
14. Collection-encoding-streaming timestamp conversion
15. Save audio and video as MP4 files
16. Collect-encode-save mp4 timestamp conversion
17. Support push screen preview
18. How to support H265 streaming
19. How to support microphone and system sound mixing
20. Implement audio collection and noise reduction
21. Will RTMP streaming cause delays?
22. How to dynamically adjust the bitrate for RTMP streaming
23. How to dynamically adjust the frame rate for RTMP streaming
24. Can the web side pull RTMP push streams?
Project significance: The streaming module of this project uses librtmp FFmpeg API. Master the streaming framework and video recording framework, including audio and video collection, module division, Core technologies such as audio and video synchronization, encoding optimization, and push delay.
4.3 FFmpeg RTMP streaming recording_0voice low-latency streaming live project
01. Introduction to RTMP protocol
02. Streaming video frame analysis
03. Pull module packaging
04. AAC decoding module packaging
05. H264 decoding module packaging
06. Packet queue design
07. Frame queue design
08. Realize audio and video synchronization
09. Is it possible not to do audio and video synchronization analysis?
10. Can it be played without MetaData?
11. Will RTMP streaming cause delays?
12. How to detect RTMP streaming delay
13. How to solve RTMP playback delay
14. Add support for variable speed playback
15. Variable speed playback strategy analysis and settings
16. Principle analysis and implementation of instant open function
17. Can ffplay and vlc be used to test playback delay?
18. Implement streaming and save videos as MP4
19. Pull stream, save MP4 timestamp conversion
Note: Master the RTMP live streaming and recording framework, including core technologies such as streaming, queue design, audio and video synchronization, low latency, and saving as MP4 files.
4.4 RTSP streaming recording project
01. Introduction to RTSP protocol
0.2 Realize RTSP streaming and recording based on RTMP streaming project
Project significance: The streaming module of this project uses FFmpeg API. Master the streaming framework and video recording framework, including audio and video collection, module division, Core technologies such as audio and video synchronization, encoding optimization, and push delay.
4.5 RTSP streaming video project
01. Introduction to RTSP protocol
0.2 Realize RTSP streaming video based on RTMP streaming project
Note: Master the RTSP streaming and recording framework, including core technologies such as streaming, queue design, audio and video synchronization, low latency, and saving as MP4 files.
Project Practice
5.1 Self-research projects and source code analysis
01. SDL audio playback
02. SDL video playback
03. FFmpeg memory reference counting
04. MP4/FLV/TS demultiplexing
05. Extract H264
06. Extract AAC
07. H264/H265 decoding
08. AAC/MP3 decoding
09. AVIO memory input mode
10. Audio Resampling
11. Video size format conversion
12. DXVA2 decoding
13. NVDEC decoding
14. CUVID decoding
15. QSV decoding
16. AAC encoding optimization
17. H264 encoding optimization
18. H265 encoding optimization
19. AAC to MP3
20. H264 to H265
21. DXVA2 hardware encoding
22. NVENC hardware encoding
23. CUVID hardware encoding
24. QSV hardware encoding
25. MP4 synthesis
26. FLV synthesis
27. Multi-channel audio mixing
28. Add watermark to video
29. Video area cropping
30. Video flip
31. Add logo to video
32. ffplay player
33. Self-developed QT player
34. RTMP streaming
35. RTMP streaming
36. RTSP streaming
37. RTSP streaming
38. HLS streaming
Silent service
6.1 Suitable for engineers
01. On-the-job engineers who are engaged in audio and video job development but do not have time to study systematically
02. Working engineers who are engaged in embedded development and want to switch to audio and video development
03. Working engineers who are engaged in desktop development such as Qt/MFC and whose salary has not increased much over the years.
04. Working engineers who are engaged in Android/IOS mobile development and want to develop in the audio and video direction
05. Working engineers who are engaged in C/C backend development and want to develop in the direction of streaming media servers
06. Developers who are slow in their own research and learning and cannot systematically build audio and video knowledge systems
07. Computer-related majors who want to enter a large factory (bachelor degree or above)
6.2 Teaching methods
01. 80 hours course, support FFmpeg6.0
02. Teacher Q&A, work issues, course issues
03. Homeroom Supervision Homework Statistics Blog Statistics
6.3 Job-hopping during interview
01. Resume sorting, highlighting technical points, sorting out project technology
02. Mock interview Technical description Project focus
03. Salary Negotiation, Struggle for Benefits, Company Negotiation
04. Offer selection, career planning, technical prospects
6.4 Learning methods
01. Read the documentation first and then the video
02. First run and debug the course code, then modify and debug it, and finally try to rewrite it yourself
03. Summarize the knowledge points of each class into a technical blog (there is a monthly blog competition with bonus support)
04. Combine courses with work and rethink demand functions
05. Communicate with the teacher about technical points you don’t understand.
06. Partial audio video client direction: Server-related content is mainly about understanding. For example, SRS streaming media server, it is mainly about understanding the push-pull streaming framework and protocol conversion logic. There is no need to invest too much time in reading the source code.
07. Streaming media server direction: The design of the UI interface is based on understanding. For example, the player project only needs to focus on audio and video related content.
6.5 Solve the difficulties of audio and video self-study
01. It is very difficult to self-study audio and video, and it is difficult to find easy-to-understand information on the Internet.
02. The online information version is older, such as FFmpeg3.x version
03. Audio and video involve a large number of open source libraries and protocols, which are difficult to compile and understand.
04. Some embedded audio and video friends have narrow technical skills and do not have deep technical skills.
6.6 High-paying employment
01. Student offer display link
6.7 Advanced extensions
0.1 Streaming media server further supplements learning
0.2 WebRTC further supplementary learning