Architectural Optimization Of Large Scale Astronomical Data

Satellites all around the globe generate huge quantities of data periodically on a daily basis. The management of such large quantities of data poses significant problems ranging from data storage, retrieval and manipulation. The current scenario of managing Indian Space Research Organisation's (ISRO) ASTROSAT-CZTI satellite data involves an elementary and unsophisticated approach which causes a lot of delay in data retrieval and further causes noticeable data portability issues. The evaluation and research on the above mentioned complication paved a way for us to introduce a new architectural system to solve this complex optimization problem. We propose the creation of an independent application server and an underlying Django REST API based web service which would facilitate the efficient management and retrieval of this data. This also extends the scope, making the system more dynamic and helps in load balancing. The application server helps the system to regulate the turbulent flow of huge data quantities. The Django web framework provides feature distribution which helps manage unorganised data cost effectively. Thus, it removes the possible causes of deterioration of response in the existing system. The REST API extends the scope of the system by providing umpteen useful features which makes universal interfacing of our system possible. The techniques used in the development of our proposed system including data migration, prefetching of data and minimizing the time trade-off, address all existing challenges and show a significant improvement in the overall performance of the system.