Please read the late submission and collaboration policy on the course website:
This problem set is provided as a ZIP file. You are reading the main problem set PDF in the root pset2/ directory. Look in the pset2/code sub-directory for code for different problems, with data in pset2/code/inputs. You will be required to modify and fill-in various functions in the .py files in the code directory. Running this code (e.g., with python ./prob2.py and so on) will create output images in the pset2/code/outputs directory.
Complete the code files in pset2/code/ by filling out the required functions.
Run each program to generated output images in pset2/code/outputs directory.
Create a PDF report file created with LATEX. Use the template provided on the course website (scroll down to the resources section at the bottom). In particular, make sure you fill out your name/wustl key (top of the .tex file) to populate the page headers. Also fill out the final information section describing how long the problem set took you, who you collaborated / discussed the problem set with, as well as any online resources you used.
The main body of the report should contain responses to the math questions, and also include results and figures for the programming questions as asked for. These figures will often correspond to images generated by your python code in the pset2/code/outputs/ directory.
Place this report file as solution.pdf in the root pset2/ directory.
Then, zip all the contents and sub-directories of pset2/ into a single ZIP file, and upload it to blackboard.
A traditional approach to image denoising is to apply what is called L1 regularization to wavelet coefficients. Since the wavelet transform is unitary, additive un-correlated Gaussian noise to image pixel intensities shows up as un-correlated Gaussian noise, with the same variance, in wavelet coefficients. If our regularization is applied independently to wavelet coefficients, then we can carry out independent minimizations for each coefficient.
What value of the scalar variable x minimizes the following cost function: x = arg min(x y)2 + jxj
Here, y is also a scalar corresponding to the noisy observation, is a parameter, and jxj refers to the absolute value of x. Find the expression for x in terms of y and . Remember that the gradient of j j is discontinuous at 0, and changes sign. (10 points)
Implement this expression in the function denoise_coeff in prob1.py. The function will be called with an array of “noisy” coefficients corresponding to y above, and a value of . Return an array of the corresponding clean coefficients, based on the above expression (applied to each coefficient independently).
You will also need to fill out the im2wv and wv2im functions from problem 6 in PSET1. (If you couldn’t get that to work, we’ll provide you with a working version of those routines). The support code handles the task of loading a noisy image, doing the pyramid decomposition, calling the denoise_coeff function with the ’right’ value of for each level, and saving the denoised image in outputs/prob1.png. Include the denoised image in your report. Feel free to play around with modifying the values of being used by the support code. (5 points)
PROBLEM 2 (Total: 15 points)
In this problem, we will implement two strategies to doing color constancy, or white balance in the file prob2.py. In each case, you will correct an image by multiplying the red, green, and blue channels with scalars r, g, and b, which will be normalized so that r + g + b = 3.
Gray World: Fill out the balance2a function to do white balance with factors r, g, and b computed to be inversely proportional to the mean red, blue, and green intensity in the image. Running prob2.py will generate white-balanced versions outputs/prob2a_?.png of the images in inputs/CC/. Include these images in your report. (5 points)
Implement the balance2b function to now compute the factors r, g, and b to be inversely proportional to averages computed over the 10% brightest intensities in each channel (i.e., r is inversely proportional to the average of the 10% highest red intensity values, and so on). The function np.sort may be useful here. Run prob2.py to get the corre-sponding versions outputs/prob2b_?.png. Include these in your report, and comment on how they differ from the results from part (a). (10 points)
PROBLEM 3 (Total: 20 points)
Implement photometric stereo as discussed in class for Lambertian surfaces with known light from point sources at infinity.
(a) Fill out the pstereo_n function in prob3.py to compute surface normals given a set of color images, and a matrix
of lighting directions. As described in class, do this by first converting the images to grayscale (you can simply take mean or sum of R,G, and B values) and then solving for the set of `Ti n = Ii = (Ri + Gi + Bi) equations at every pixel. For this part, you are allowed to use np.linalg.solve, but DO NOT use linalg.lstsq or any pre-programmed least-squares solver. Implement it yourself. (Note also that the function is provided with a mask of where pixels are valid. Outside the valid region, you may get division by zero errors when you try to normalize the normal vectors, etc. Be sure to avoid this.)
Include the saved image of your estimated normals outputs/prob3_nrm.png in your report. (10 points)
(b) Fill out the pstereo_alb to now compute an image of the surface color albedos, given the computed normals from the previous step. (This is your solution for R`Ti n = Ri, and so on.) The results of this will be saved as outputs/prob3_alb.png. Include it in your report. (10 points)
PROBLEM 4 (Total: 25 points)
Implement the ntod function in prob4.py to compute a depth map, i.e. an array of surface height values Z, from estimated surface normal vectors, in the Fourier domain using the Frankot-Chellappa method as described in class. Your function will be called with the array of normal unit vectors, a mask where these values are valid, and regularization weight. Set the x and y derivatives to n^x=n^z and n^y=n^z where the normals are valid, and to 0 where they are not (i.e., where the mask is 0). Use the same regularizer as described in class (in terms of the fr filter) weighted by the regularization parameter passed to the function.
Running the program will generate a python plot window with a 3D surface plot of the surface. Rotate it interactively to what you think is a good view, export the figure as a png (click on the Save button on your plot window), and include it in your report.
PROBLEM 5 (Total: 25 points)
Implement the ntod function in prob5.py, now using conjugate gradient. This function takes the same parameters as in prob5.py and minimizes a similar cost, except that the squared penalties for the derivatives are now weighted differently at each pixel as discussed in class. The weight is set to 0 at invalid pixels (where the mask is 0), and to (^nz)2 everywhere else. Note that there is no per-pixel weighting on the regularizer—just , the third parameter passed to the function, at all pixels.
Once your function is working, setup your code to run conjugate gradient for at least a 100 iterations (you can use fewer iterations while debugging). Like for the previous problem, running the support code produces a 3D surface plot. Export this figure as well, and include it in your report.