[Solved] AMATH482 Assignment5-Background Subtraction in Video Streams

Use the Dynamic Mode Decomposition method on the video clips ski drop.mov and monte carlo.mov containing a foreground and background object and separate the video stream to both the foreground video and a background.

The DMD spectrum of frequencies can be used to subtract background modes. Specifically, assume that _p, where p {1,2,,`}, satisfies k_pk 0, and that k_jk j 6= p is bounded away from zero. Thus,

XDMD = bppept + Xbjjejt (1)

| {z }

Background Video

Foreground Video

Assuming that X R^nm, then a proper DMD reconstruction should also produce X_DMD R^nm. However, each term of the DMD reconstruction is complex: b_jj exp(_jt) C^nm j, though they sum to a real-valued matrix. This poses a problem when separating the DMD terms into approximate low-rank and sparse reconstructions because real-valued outputs are desired and knowing how to handle the complex elements can make a significant difference in the accuracy of the results. Consider calculating the DMDs approximate low-rank reconstruction according to

XLow-Rank_DMD = b_ppe^pt.

Since it should be true that

X = XLow-Rank_DMD + XSparse_DMD ,

then the DMDs approximate sparse reconstruction,

XSparse_DMD = Xb_jje^jt,

j6=p

can be calculated with real-valued elements only as follows

SparseLow-Rank

XDMDXDMD ,

where | | yields the modulus of each element within the matrix. However, this may result in X^Sparse_DMD having negative values in some of its elements, which would not make sense in terms of having negative pixel intensities. These residual negative values can be put into a nm matrix R and then be added back into X^Low-Rank_DMD as follows:

XLow-Rank_DMD XLow-Rank_DMD

XSparse_DMD XSparse_DMD R

This way the magnitudes of the complex values from the DMD reconstruction are accounted for, while maintaining the important constraints that

X = XLow-Rank_DMD + XSparse_DMD ,

so that none of the pixel intensities are below zero, and ensuring that the approximate low-rank and sparse DMD reconstructions are real-valued. This method seems to work well empirically.

1