Once an optical imaging system is implemented, several phases of hardware modification/optimization occur. Eventually high quality images are produced that represent the morphology or function of a tissue/organ. Since the images are generated via a complex light-tissue interaction, they contain a lot more information. We have studied the light-tissue interaction using Maxwell equations and other approximations and modelled the tissue behavior. We then developed several statically-driven methods to extract quantities and associate them to the tissue status. With such quantities, we differentiated between diseased and healthy tissue. We also utilized the power of the priori information and learned the pattern of the disease utilizing supervised methodologies such as machine learning. We have explored the power of image analysis (with / without the use of machine learning) in both optical coherence tomography and photoacoustic imaging. We believe that hardware improvement has a limit and the more improvement in the diagnostic outcome of an imaging system can be achieved using signal/image analysis.