Image Editing from multiple photos to one single best

Abstract

These instructions are for the Demo, the procedure are the same for the web App. To Explain the user interface we will take a step by step walk through.

Step 1 Select reference view

Select your choice of reference image by scrolling the mouse wheel inside the "mosaic view" in this way you can traverse multiple images. To select one base reference image simply click in that highlighted image.

Step 2 Global thresholding

The difference between the reference image and all other images are computed and highlighted, these highlighted areas are called layers. This allows you to select content to add or remove. Adjust the Global threshold by scroll the mouse wheel in anywhere except in the green one layer, this is done to split or merge the layers with each other. The green is explained in next step.

Step 3 Toggle

When some content that you want to change is present click in that layer to make changes in the image. You can toggle the content of that specific layer by click in it.

The green layer is just an indication that the layer is selected. The content inside the green layer are coming from an image other than the reference image. The content inside all other colored layers then the green mean that that content are coming from the reference image. Different colors means that in that area there is differences with respect to the reference image. The reference image in this example only have one other image then the reference image so there is only one blue color indicating differences between these two. To choose a green layer means that when we hit the blend button then the script will automatically blend the content we see inside the green layer into the reference image.

Step 4a Fine tune Relative threshold

Adjust the Relative threshold by scroll the mouse wheel inside any green layer, the layer you are currently hovering are highlighted by a light-green color to indicate which region you are adjusting.

To catch up with Step 2 - 4a , these include handling of mouse scroll wheel, mouse point and mouse click when the check box Select Layer is selected. For a comprehensive view see the flowchart.

Step 4b Fine tune Paint/Erase

If you are not satisfied with what the layer regions are covering you can manually paint or erase the layers by selecting these options in the check-box. Simply paint inside a green region to add layer.

Erase parts of the layer by first select the eraser to the left.

Step 5

Click the blend button to let the application blend in all selected layers in the reference image. Please be aware of that this is an experiment and not a fully fledge app.

Use Case Examples

Remove Content

We have two photographs taken from a sequence of a two person portrait in a corridor seen in the figure above to the left. We decide that we only want a portrait of the person in the the blue shirt. Ideally we want to combine the exposure b down-left where we have that person looking more relaxed then on exposure a top-left, there is also other details like the photographer behind our protagonist who accidentally ended up in frame a. So in other words we want to remove the girl from image frame b. To do this we combine the two exposures and replace the girl to the right in frame b with the background wall from frame a to produce the Output result seen in figure c.

Add content

We have three photos taken from the same spot at slightly different times which essentially resulted in us capturing multiple cars. We have photographies of cars but in several different frames. The preference could be one single photo where as many cars as possible appear visible.

Our goal is to put together details from multiple images into one single frame. Here we are adding details from our pictures into one reference image. So we select one reference view that will include an area of the multiple cars, this is the middle frame which also contains a car in the middle. Now we can combine the left cars from the left frame and the right car from the right frame to create the concise photo we are looking for, see figure d

Duplicate Content

Here we want to capture the motion of two objects namely the skater and the skateboard in one single frame. The process is very similar to the previous example we are adding up details from several different pictures into one single frame, but the difference is that it is the same objects we are adding. Thus in this example we duplicate the two objects from our set of frames into one single frame.

With this technique one can use several images to, for example "clone" people within a scene and thereby visualize their path of motion in one single-frame. Of course, the conditions are that we have appropriate data suited to the purpose in form of multiple photographies of the same scene and also taken from more or less the same viewpoint. And preferably, the scene in question should contain some form of dynamics.

Introduction to Burst

Burst mode is a good choise when it comes to photographing moving subjects. One example is shooting sports. Sports photography can be hard because sports move fast, the athletes move fast and the fields are big. With the camera in burst mode we can hold down the shutter and pan the camera along when the athletes movement.

Putting the camera settings in to burst mode are good in other situations too. Animals and children are similar to athletes in the way that the move fast.

You often only get one chance to capture a snapshot and to do this in a situation where many things happening at ones, to freezing the moment of a dynamic scene when everything looks right can be a harsh task.

The success rate is usually low and therefore photographers need good software to to edit down the number of candidates to do post-processing in. It is here our proposal comes into play.

Magcut is an image editor that let the user rearrange the photographed scene to maximize the visibility or to change the composition. This can be done in several ways either by adding and moving objects or by removing objects within the scene.

Our application is suited for dynamic scenes where objects are moving. It also allows the camera to move slightly within a scene. We assume that the burst of images provided by the user contains some differences. That is, differences between each single image.

Materials

The source code and the data can be found on github in the Link.

Related

The application uses image stitching. Please see The project Image Stitching with HTML5, canvas and JavaScript Demo page and/or the Source code for more information on stitching.