Hello! I’m Madhur Sudarshan, a final year Masters student conducting research in the fields of Computer Graphics and Computer Vision (i.e. broadly Visual Compute). I will graduate with a Masters and Bachelors in Electrical Engineering and a minor degree in Computer Science (expected July 2022) from Indian Institute of Technology Bombay. I am advised by Prof. Parag Chaudhuri as my guide and Prof. Subhasis Chaudhuri as co-guide. I also work with Prof. Sharayu Moharir and have worked with Prof. Nikhil Karamchandani.

I am passionate about enabling Immersive Technologies and understanding their use for environmental story telling. My current research focuses heavily on XR, and my career goal is to create tools for Artists and platforms for Professionals.

I am tackling Coherent Illumination for Mixed Reality for my masters thesis and Deep Reinforcement based policies for Edge service caching with a focus on edge Cloud based VR processes as an additional research project. I have also worked on Audio Processing, creating a novel framework for paired instrument transfer using Wavenet. For more information on my projects please refer to the refer to my CV or view a summary of significant research projects below.

Key Research Projects

I am currently working on enabling tools for high realism in Augmented Reality in two projects-

• Coherent Rendering for Augmented Reality (Advised by Prof. Parag Chaudhuri and Prof. Subhasis Chaudhuri) (Ongoing)

  Synthetic objects rendered onto real environments are not influenced by lights and reflections in the real world. Our eyes are perceptive to these inconsistencies, thus to render objects coherently we must infer accurate illumination information. We are interested in inference approaches that run in real time on monocular RGB cameras, that are available on most handheld devices. I began by recreating the paper titled Learned Light Probes for Mixed Reality Illumination (Mandl et. al.) that trained a CNN per pose to extract spherical harmonic representations of illumination from images of an object with known geometry, textures and BRDF. I synthesized datasets for objects to test this approach virtually. I explored augmentations to and identified fundamental drawbacks with their approach. Inspired by recent research in Differentiable Rendering and Neural Rendering I tried to generate intermediate images for realtime applications, which brought me full circle to older approaches on inverse lighting. I implemented an app to run this for known poses in Unity, and am currently working on improving that experience. I hope to publish this work soon.

  

• Deep Reinforcement Learning for Service Caching on the Edge (Advised by Prof. Sharayu Moharir and Prof. Nikhil Karamchandani) (Ongoing)

  Enabling High Quality XR apps on Head Mounted Devices provides issues with Compute and Battery Limitations. Instead of running processes locally hosting such services on edge servers and streaming to mobile devices is a promising alternative. There are also many research challenges in ensuring high Quality-of-Experience to users, typically seen in terms of latency and bandwidth limitations. Taking inspiration from Predictive Scheduling for VR, I completed an extensive literature survey and proposed an alternate QoE based metric to optimise the choice of running services locally or on edge servers. Then to investigate performance of reinforcement learning policies against optimal solutions, I am studying the performance of deep reinforcement learning agents on long horizon arrival patters and loss metrics on an approach titled RetroRenting by my advisors and V S Ch Lakshmi Narayana. When this research is concluded insights from this can be applied to other optimisation problems for edge cloud based services.

Aside from these I also have a strong background in Audio Processing, both as an Electrical Engineer/AI Researcher and Musician/Producer. Of particular interest would be my research conducted as a course project for Advanced Machine Learning-

• Wavenet for Timbre-Transfer (Team-mate: Harsh Prashant Dolhare, Instructor Prof. Amit Sethi)

  Musicians performing live seek to increase their sonic pallete by using many effects such as filters, distortion pedals. A more challenging problem is to make one instrument sound like another, this is “timbre transfer”. The instrument that offers the most variation in timbre would be electronic instrument like the keyboard which can emulate any sound, but these restrictive in expressivity. More expressive instruments such as wind or string based instruments conventionally need frequency information to be converted into MIDI information before they can generate varied sounds but this can lead to artifacts. Borrowing from research on modelling distortion pedals as a blackbox using Wavenet, I proposed a model for paired instrument transfer that can run in realtime on any input instrument to map to other output instruments. We generated novel datasets and study the performance of this model. This approach has promising results, though it is unable to recreate stochastic note onsets and cannot map instruments with shorter spectral range to ones with wider spectral ranges. The results can be seen here

I also spent a summer researching the application of High-Order Singular Value Decomposition to Liquid Chromatography-Mass Spectrometry data for metabolomics. My challenge was to apply Image Processing concepts to sparsely encoded data, with an end goal of processing many Gigabytes of unanalysed data. This work was done with Prof. Pramod Wangikar for the start-up Clarity Biosystems. Aside from this course projects can be found in my CV.

Background

For the majority of my life IIT campus in Mumbai has been my home, with the exception of a pleasant kindergarten year in Seattle Washington in the Redmond area, and a blissful exchange Semester in DTU Copenhagen, Denmark. I grew up surrounded by academics in IIT Bombay where my father Prof. S. Sudarshan is a Professor of Computer Science. This environment inspired and led me to spend my youth representing my school at various Olympiads and Quizzes. I was also fortunate to pursue art classes, various sports and robot coding bootcamps. My first exposure to professional instruction was a summer transcribing and dubbing coding tutorials for Spoken tutorials. That and a fascination with Science Fiction Literature ranging from Isaac Asimov to Douglas Adams brought me to pursue a Degree in Engineering. My background as a musician and an artist ensured that I stuck close to subjects that were creative in nature, thus leading me to pursue problems related to Computer Graphics as my Masters thesis.

Extra-Curricular Pursuits

I’m a semi-professional session musician in my free time, and this forms a major motivation for pursuing research too. I began learning music at the age of 9, covering various musical instruments across my journey ranging from Keyboard, Drums and Singing, but I finally settled on the bass guitar as my primary gigging instrument. I enjoy improvisation and performing jazz fusion music, though I have been fortunate to play with and lead Progressive Rock, Jazz and Funk bands. As a bandleader I’ve led my group to perform infront of audiences of over 5000 opening for popular Bollywood singers. I’ve completed courses on Music Theory, Linguistics and Industrial Design and am particularly interested in how different forms of media reflect the political and cultural backdrop of their times. This reflects in my Research Interests in semiotics and perception. Besides this I used to play Basketball Representing my University and School. More recently on a semester exchange in Denmark I was able to realise an unfulfilled passion for cooking. Besides having a stellar social experience I was also introduced to knitting, a hobby I hope to reconnect with if I move to colder climates than Mumbai :)