This is a technical intro to the process of creating 360 videos by stitching several viewpoints of the same scene into a single video that can be viewed with VR goggles, or in a 3D app.
The high-level process goes something like this:
- Figure out how the cameras are oriented relative to each other.
- Determine which camera should be used for a pixel when there are overlapping areas between cameras.
- Adjust images to account for lens aperture.
- Blend images, avoiding any visible seams.
- Compensate for varying camera exposures (The video from one camera may be lighter).
- Map the processed images into a single-image “map” that a VR or 3D app can render.
- Encode this map in a video encoder.
Where to go from here
The tutorials here describe the 360 video stitching process used in OpenCV, but does not attempt to teach the mathematics and algorithms required.
Udacity has a good computer vision course (free) that goes through each of the topics needed for 360 video stitching (minus the video encoding). Linear Algebra is a pre-requisite for most computer vision topics, including this course.
One of the best books on Computer Vision is Computer Vision: A Modern Approach by Forsyth & Ponce. The Table of Contents and Preface is available for free if you want to get a sense of the topics, but the book has to be purchased (disclaimer: we receive a commission if you buy the book through the link). There are digital and physical versions available.