PTStitcherNG 0.7b

Prof. Dr. H. Dersch - HFU Furtwangen
March 6, 2010

Introduction

PTStitcherNG reads and transforms any number and type of input images, and combines them into one seamless panoramic image with larger field-of-view. Given enough input images, full spherical 360x180 degree views can be synthesized suitable for virtual-reality viewers or printing. Transformation parameters and names of the input images are read from a plain-text projectfile. The transformations consist of correcting distortions due to camera lenses, perspective tranlation and remapping to any of the usual panorama projections. Merging employs an eight level multiresolution algorithm which hides seams even if the source images fit badly. PTStitcherNG natively reads PPM, TIFF and JPEG images, and almost any raw or other format through plug-ins.

PTStitcherNG combines a panorama stitcher and multiresolution blender in one application. It is optimized for speed by using parallel processing (SIMD-instructions, multiple processor cores). The main novel feature is the tight coupling of the remapper and blender, with an optimized management of temporary data. This enables PTStitcherNG to keep all intermediate data and processed images in ram (in the case of the CUDA-version: in gpu-ram) even when stitching hundreds of images to gigapixel sized panoramas. As a result, no data have to be written to disk, or reread from disk with corresponding speed improvements. As an aside, this makes PTStitcherNG perform fast even on low profile host systems

Some speedresults, obtained with my fastest platform (Intel Celeron 1.8GHz, 1 GByte ram, Nvidia GeForce GTX 285, OpenSUSE 11.1):

3 x 11 MPixel -------> 18 MPixel equirect 2.1 sec (the decimal point is correct)
For source images and project files see here.

337 x 6 MPixel ------->1433 MPixel cylinder 250 sec (4min 10 sec)
For source images and project files see here.

The cellprocessor version (Playstation 3) has been discontinued: Recent decisions by Sony (new PS3 without Linux) and IBM (dim future for the cellprocessor) made me switch to CUDA as one basis for PTStitcherNG. Friendly folks from Nvidia sent me one of their gpu-cards for evaluation. However, PTStitcherNG also runs very fast on plain vanilla current x86 systems.

PTStitcherNG is free software. No license is required to download and use it. See the licenses for auxiliary components in the Licenses folder.

Installation

All versions of PTStitcherNG are copy-installed, i.e. the necessary program files are simply copied to a suitable location on your computer. No automatic "installer" corrupts your system by manipulating low-level settings.

Windows

The Windows version of PTStitcherNG has now been tested to work on Windows 7. Binaries for 32bit and 64bit platforms, with and without CUDA are provided. Use one of the files PTStitcherNG_cuda.exe or PTStitcherNG.exe. PTStitcherNG.exe should run as is, while the CUDA-version requires two libraries (dlls): The file pano_cuda.dll from the distribution, and Nvidias CUDA-runtime cudart.dll which is now included in the distribution. You also need Nvidias CUDA-driver (and, of course, a CUDA-capable grapics card): Download from Nvidia and install it.

X86-MacOS

Only the non-CUDA version of PTStitcherNG is available for Macintoshs. Simply copy the files PTStitcherNG (the actual program) and the file PTStitcherNG.app (an AppleScript GUI-wrapper for PTStitcherNG) to your computer. The default PTStitcherNG application is compiled for MacOS 10.5 (Leopard). There is no version for older versions of MacOS or PowerPC processors.

Extending PTStitcherNG

As is, PTStitcherNG reads TIFF, JPEG and PPM-images. Reading RAW and other formats is accomplished by external readers, which feed PPM-data to PTStitcherNG. Once setup, this is done automatically. On startup PTStitcherNG reads a textfile PTImageReader.txt, which contains fileextensions and program commands, see the example of the distribution. Download and install dcraw for reading almost any RAW format, pngtopnm for reading PNG images, or any other of the netpbm utilities for almost any format. The distribution now includes dcraw. Just move it to the same folder as the PTStitcherNG executable together with the PTImageReader.txt file. Edit PTImageReader.txt to set special dcraw options, e.g. for using 16bit-pixels etc.

PTStitcherNG writes TIFF (multilayer using "tiff_m" and single layer "tiff", see scriptsyntax below), PPM ("ppm"), PSD / PSB and JPEG images. Pixelsize defaults to 3-byte RGB. The commandline option -p switches output format to 6-byte RGB (only PPM and TIFF). Large files exceeding 2GByte size can be written in PPM and (flat) TIFF-format, provided the operating system and file format permits it. In the case of TIFF-multilayer format, a filename with extension ".btf" or ".tf8" has to be used to force BigTIFF format on Linux and MacOS, the Windows-version supports multilayer format only for regular TIFFs.

Using PTStitcherNG

PTStitcherNG can be run as standalone programs with minimal graphical user interface. Simply drag and drop a suitable projectfile (e.g. the speedtest.pts-files from the above mentioned speedtests) onto PTStitcher's icon (in the case of MacOS: onto PTStitcherNG.app's icon). You will be asked for a save-location, and PTStitcherNG will then proceed and convert the input images.

PTStitcherNG can be run as a console application from the command line with many more options to finetune the conversion process. This might be used for batch-processing large amounts of images using shell scripts. See the man-page ptstitcher_man.pdf for details.

Finally, PTStitcherNG can be registered into one of the many graphical frontends to panorama tools, see the docs of your favorite system for details.

Examples

The distribution includes a small sample project which can be used to test the installation. Open the folder test and locate the file project.txt. Drag and drop this file onto PTStitcherNG's icon (on Macs: on PTStitcherNG.app's icon). PTStitcherNG should start up, ask for a save-location and convert the 6 images to a cylindrical panorama. Open the file project.txt with any plain-text editor, and change the settings to test other features, see the documentation on scripting syntax below. To run PTStitcherNG from the commandline, open a console window, navigate to the test-folder, and issue the command
..\win\PTStitcherNG.exe -o pano.tif Project.txt on Windows
../mac/PTStitcherNG -o pano.tif Project.txt on Macs
In this mode all commandline switches can be tested, see the PTStitcherNG manpage.

For larger panoramas and to test image quality, download any of the speedtest examples mentioned above: 18 megapixel sphere or 1433 megapixel cylinder .

Projectfile

Projectfiles are plain-text files generated either by hand using any texteditor or (usually) using one of the panorama tools utilities for aligning images, e.g. PTOptimizer, which is integrated in most graphical frontends to panorama tools. A subset of PTStitcher and PTGui scripting parameters is supported:

One line describing the output panoramic image:
p wwidth hheight fprojection vfield-of-view nfileformat
projection is one of 0 (rectilinear), 1 (cylinder), 2,4 (equirectangular), 3 (spherical equidistant), 5,10 (spherical equisolid), 6 (spherical mirror). Two formats provide perspectively corrected projections for large fields-of-views: The multiple-rectilinear format (11), for examples see here, and the Panini-format (19). Both require additional parameters which are documented in separate articles.
fileformat = PPM, PSD, PSB, JPEG, TIFF, TIFF_m (TIFF-multilayer). In the case of JPEG-ouput, the quality (q=0...100 and progressive-flag g=0,1 can be set in a quoted string like n"JPEG q95 g0". TIFF files are limited to approximately 2 Gigabyte filesize, except if the BigTIFF extension is selected. This is done either in the script by specifying b1 as option to TIFF or TIFF_m like n"TIFF b1". If standard, non-layered, (and non deferred, see manpage) TIFF-output is selected, PTStitcherNG automatically chooses this format if image size exceeds this limit. However, not all current TIFF-readers are capable of decoding this extension correctly.
Choosing TIFF_m (TIFF-multilayer) format disables the blender. The output file consists of a multiframe image with one remapped frame per input image. The frames are tagged with their offset coordinates in the global panoramic image. Using this option automatically disables blending over zenith/nadir (options -n and -z), and option -d (deferred merging).

Some notes on Photoshop native formats (PSD - up to 30000 pixels wide, PSB - up to 300000 pixels wide): Both can be 8 or 16 bit color depths. In default mode a multilayer image is created with each layer hosting one fixed-length rectangular tile. This tiled scheme is choosen for fast reading and writing access, and has no resemblance to the original source images. If so desired it can be collapsed in Photoshop onto the backgound layer without any editing or blending.
When using the '-d' (deferred merging) option, a flat image without layers is created. Note that as mentioned elsewhere, this option requires buffering of the panoramic image in memory and may not work for very large images on a low RAM system.
The PSD- and PSB-formats have been tested to work in Photoshop CS4, but many third-party programs do not support them or only some of their features. The MacOS image previewer does not correctly display the multilayer variant, only the flat version. The gimp only recognizes PSD, not PSB images. Also, multilayer 16bit-colordepth images are not rendered correctly on this platform.

and one line describing each input image:
o yyaw ppitch rroll wwidth hheight fprojection vfield-of-view nfilename aa bb cc dd ee Cx_left,x_right,y_top,y_bottom Sx_left,x_right,y_top,y_bottom
yaw, pitch and roll specify angles to tilt, pan and rotate the images.
C and S options specify rectangles: C specifies new image dimensions (crop), all other parameters like field-of-view refer to these new dimensions, while S mereley selects an area by setting the alpha channel to zero outside.
a,b,c refer to polynomial coefficients for distortion corrections, and d,e shift the optical center of the image.
The projection parameter is almost identical to the panorama case, with one exception: 2 means spherical equidistant. The spherical formats 2 and 3 are identical except that the first variant converts a circular region while format 3 converts the whole image. The same holds for formats 5 and 10.
The line describing imagefiles in PTGui-pts-projectfiles
#-imgfile width height "filename"
is also accepted to support script-generating programs which do not supply the filenames for input images on the o-line (see above). PTStitcherNG requires the input image names in the script. They can not be supplied on the commandline as in old PTStitcher. Most other parameters are optional with the exceptions y,p,r. PTStitcherNG supports the global image description line termed "#-dummyimage" in PTGui-projectfiles. Identical parameters for all images, e.g. field-of-view (v) or correction parameters (a,b,c) can be specified as v=0 (meaning identical to the value of dummyimage Nr.0 ), a=0 b=0 etc.

and one line describing global options:
m iintrepolator
with interpolator = 0(cubic, default),1 (spline 4x4), 2 (spline 6x6), 3 (sinc 16x16), 4 (spline 8x8), 5 (bilinear), 6 (nearest neigthbor), 7 (sinc 32x32), 8 (sinc 6x6), 9 (sinc 8x8).

Viewpoint correction is supported using PTGui-pts-format:
#-viewpoint Vx Vy Vz pan tilt
This correction is a combination of lateral translation (Vx,Vy), size change (Vz), and two-axis rotation (pan, tilt).

Inverse Mode

Using the command option -i name instead of the usual -o name to specify the panorama image name switches PTStitcherNG into inverse mode. In this mode the panorama image must exist and all images specified in o-lines are created. All sensible options are supported including all warp transformations, radial distortions and perspective transformations. The fileformat specification on the p-line (tiff/jpeg/etc) is applied to the output images.
Caution: Using PTStitcherNG this way with an existing projectfile previously used for panorama-creation overwrites all source images if they exist!

There are numerous applications for this mode: Create 6 cubic faces from an equirectangular source for cubic panorama viewers with this script: p f2 v360 n"JPEG g0 q100" o f0 y0 r0 p0 v90 w1500 h1500 n"forward.jpg" o f0 y90 r0 p0 v90 w1500 h1500 n"right.jpg" o f0 y-90 r0 p0 v90 w1500 h1500 n"left.jpg" o f0 y-180 r0 p0 v90 w1500 h1500 n"back.jpg" o f0 y0 r0 p90 v90 w1500 h1500 n"up.jpg" o f0 y0 r0 p-90 v90 w1500 h1500 n"down.jpg"

Sequences of images to be merged in motion images can be extracted in a similar way to the mpremap project.