Programming
How do I export UIImage array as a movie
Transforming a collection of static images into a dynamic video is a common requirement in many iOS applications, from social media apps to creative editing suites. When you need to export UIImage array as a movie, it might seem like a complex task, but with the right approach and understanding of Apple’s powerful AVFoundation framework, it becomes a streamlined process. This guide will walk you through the essential steps, best practices, and technical considerations to efficiently convert your sequence of images into a high-quality video file, ensuring your app delivers a polished and engaging user experience. Whether you’re building a time-lapse feature or an animated story, mastering this capability is crucial for enhancing your application’s multimedia functionality and delighting your users.
Understanding AVFoundation for Video Creation
AVFoundation is Apple’s comprehensive framework for working with time-based audiovisual media. It provides powerful and flexible tools for controlling, processing, and generating media, making it the go-to choice for tasks like exporting an array of UIImage objects as a movie. While it can appear daunting at first glance due to its extensive capabilities, focusing on specific components like AVAssetWriter and AVAssetWriterInput simplifies the task of video creation from image sequences.
The core idea behind using AVFoundation for this purpose is to treat each UIImage in your array as a single frame in the output video. You’ll write these frames, one by one, to a video file using AVAssetWriter. This class manages the writing of media data to a new file, while AVAssetWriterInput represents a track of media data (like video or audio) that will be written to the asset. By configuring an AVAssetWriterInput for video and appending each image as a sample buffer, you effectively construct your movie frame by frame. This method offers fine-grained control over video quality, frame rate, and compression settings, which are vital for a professional output.
According to Apple’s official documentation, AVAssetWriter is designed for “writing media data to a new file,” highlighting its suitability for generating new assets from raw data or existing media. Developers leveraging this framework gain robust control over the encoding process, allowing for optimization specific to their application’s needs, such as resolution scaling or bit rate adjustments. This level of control is paramount for achieving desired file sizes and visual fidelity when creating videos from static images.
Preparing Your UIImage Array for Export
Before you can begin the actual video export, proper preparation of your UIImage array is essential. This involves ensuring your images are consistently sized, correctly oriented, and in a format suitable for video processing. Inconsistent image dimensions can lead to stretching, cropping, or artifacts in the final video, while incorrect orientation can result in upside-down or sideways frames. It’s good practice to resize and crop all images to a target video resolution (e.g., 1920x1080 for Full HD) before feeding them into the video writer.
Beyond dimensions, consider the color space and pixel format. While UIImage handles many formats internally, converting them to a consistent pixel buffer format, such as kCVPixelFormatType_32ARGB or kCVPixelFormatType_420YpCbCr8BiPlanarFullRange (for better compression), before writing to AVAssetWriterInput can improve performance and compatibility. This conversion often involves using CVPixelBufferPool and CGContext to draw the UIImage into a CVPixelBuffer, which is the format AVFoundation expects for video frames. This step is critical for efficient processing and ensures that the video encoder receives data in an optimal structure.
For applications that require dynamic content, such as generating animated GIFs or short video clips from user-created content, this preparation phase is where much of the visual quality is locked in. For instance, an app creating a “photo story” might take user photos of various sizes, but before exporting, it would scale and center each one within a standard video frame size, perhaps adding a subtle pan or zoom effect using Core Animation techniques to add visual interest. This meticulous preparation prevents common video export issues and contributes significantly to the final output’s professional appearance.
Step-by-Step Guide: Exporting Images to Video
Exporting a UIImage array as a movie involves several key steps using AVAssetWriter. This process ensures each image is correctly converted into a video frame and written sequentially to a designated output file. This method is robust and gives you control over crucial video parameters like frame rate and encoding quality.
-
Initialize AVAssetWriter and Output URL:
First, define the output URL for your video file (e.g., in the app’s temporary directory). Then, initialize an
AVAssetWriterinstance with this URL and a file type (e.g.,AVFileType.mp4). Handle any potential errors during initialization, as a failure here means the video cannot be created.let outputURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("output.mp4") try? FileManager.default.removeItem(at: outputURL) // Remove existing file guard let assetWriter = try? AVAssetWriter(url: outputURL, fileType: .mp4) else { return } -
Configure AVAssetWriterInput:
Create an
AVAssetWriterInputfor video and define its output settings. These settings include the video codec (e.g.,AVVideoCodecType.h264), resolution (e.g., 1920x1080), and pixel buffer attributes. Ensure the pixel buffer attributes match the format you’ll use for converting yourUIImageobjects toCVPixelBuffers.let videoSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.h264, AVVideoWidthKey: 1920, AVVideoHeightKey: 1080, AVVideoCompressionPropertiesKey: [ AVVideoAverageBitRateKey: 6_000_000, AVVideoMaxKeyFrameIntervalKey: 30 ] ] let videoInput = AVAssetWriterInput(mediaType: .video, outputSettings: videoSettings) let adaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: videoInput, sourcePixelBufferAttributes: nil) // sourcePixelBufferAttributes can be nil or specific -
Add Input and Start Writing:
Add the configured
AVAssetWriterInputto yourAVAssetWriter. Then, callstartWriting()on the writer andstartSession(atSourceTime:)to begin the writing session. The source time typically starts at.zero.if assetWriter.canAdd(videoInput) { assetWriter.add(videoInput) } assetWriter.startWriting() assetWriter.startSession(atSourceTime: .zero) -
Append Image Frames:
Iterate through your
UIImagearray. For each image, convert it into aCVPixelBuffer. Use theAVAssetWriterInputPixelBufferAdaptorto append this pixel buffer to the video input at a specific presentation time. Question & Answer :I have a serious problem: I have an
NSArraywith severalUIImageobjects. What I now want to do, is create movie from thoseUIImages. But I don’t have any idea how to do so.I hope someone can help me or send me a code snippet which does something like I want.
Edit: For future reference - After applying the solution, if the video looks distorted, make sure the width of the images/area you are capturing is a multiple of 16. Found after many hours of struggle here:
Why does my movie from UIImages gets distorted?Here is the complete solution (just ensure width is multiple of 16)
http://codethink.no-ip.org/wordpress/archives/673Take a look at AVAssetWriter and the rest of the AVFoundation framework. The writer has an input of type AVAssetWriterInput, which in turn has a method called appendSampleBuffer: that lets you add individual frames to a video stream. Essentially you’ll have to:
1) Wire the writer:
NSError *error = nil; AVAssetWriter *videoWriter = [[AVAssetWriter alloc] initWithURL: [NSURL fileURLWithPath:somePath] fileType:AVFileTypeQuickTimeMovie error:&error]; NSParameterAssert(videoWriter); NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys: AVVideoCodecH264, AVVideoCodecKey, [NSNumber numberWithInt:640], AVVideoWidthKey, [NSNumber numberWithInt:480], AVVideoHeightKey, nil]; AVAssetWriterInput* writerInput = [[AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:videoSettings] retain]; //retain should be removed if ARC NSParameterAssert(writerInput); NSParameterAssert([videoWriter canAddInput:writerInput]); [videoWriter addInput:writerInput];2) Start a session:
[videoWriter startWriting]; [videoWriter startSessionAtSourceTime:…] //use kCMTimeZero if unsure3) Write some samples:
// Or you can use AVAssetWriterInputPixelBufferAdaptor. // That lets you feed the writer input data from a CVPixelBuffer // that’s quite easy to create from a CGImage. [writerInput appendSampleBuffer:sampleBuffer];4) Finish the session:
[writerInput markAsFinished]; [videoWriter endSessionAtSourceTime:…]; //optional can call finishWriting without specifying endTime [videoWriter finishWriting]; //deprecated in ios6 /* [videoWriter finishWritingWithCompletionHandler:...]; //ios 6.0+ */You’ll still have to fill-in a lot of blanks, but I think that the only really hard remaining part is getting a pixel buffer from a
CGImage:- (CVPixelBufferRef) newPixelBufferFromCGImage: (CGImageRef) image { NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey, [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey, nil]; CVPixelBufferRef pxbuffer = NULL; CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, frameSize.width, frameSize.height, kCVPixelFormatType_32ARGB, (CFDictionaryRef) options, &pxbuffer); NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL); CVPixelBufferLockBaseAddress(pxbuffer, 0); void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer); NSParameterAssert(pxdata != NULL); CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB(); CGContextRef context = CGBitmapContextCreate(pxdata, frameSize.width, frameSize.height, 8, 4*frameSize.width, rgbColorSpace, kCGImageAlphaNoneSkipFirst); NSParameterAssert(context); CGContextConcatCTM(context, frameTransform); CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image), CGImageGetHeight(image)), image); CGColorSpaceRelease(rgbColorSpace); CGContextRelease(context); CVPixelBufferUnlockBaseAddress(pxbuffer, 0); return pxbuffer; }frameSizeis aCGSizedescribing your target frame size andframeTransformis aCGAffineTransformthat lets you transform the images when you draw them into frames.