864 / 2019-04-30 10:02:22

Research on Big Data Key Technologies of Real-time Processing of Power System Massive Log Data

Big Data, Massive Log Data, Power System, Real Time Processing, Smart Grid

With the development of power network information, information technology has penetrated into all aspects of power production. The power information system generates a large amount of log data during the running process, including system network connection status, database status and other system operation logs, and service related logs such as device running parameters, and they are very import to modern smart grid. This paper studies the key technologies of power data system log big data collection and preprocessing, proposes the collection method of log data of different types of information systems based on Flume and other data acquisition algorithms, and designs a optimized real-time analysis and processing architecture for massive log data of multi-source, heterogeneous and multi-time windows. The proposed approach is verified numerically and experimentally to be accurate and reliable.