Abstract:

With the vigorous development of big data researches, the related technology products are becoming more and more complicated. This paper first summarizes the general processing flow of large data composed of data acquisition, data preprocessing, data storage and management, data analysis and data display, and then combs the existing big data technology from various stages of the process, classifies and analyses the primary technical products. Besides, we also discuss and summarize the problems and future development trends of each stage to establish a big data technology panorama view from the perspective of process flow, and provide guidance for the development and application of big data technology.

Key words: Big data, Processing flow, Technology product, Open source, Artificial intelligence, Privacy security