‘LuoJiaNET’, the world’s first dedicated deep learning framework for remote sensing image intelligent interpretation, and ‘LuoJiaSET’, the industry’s largest remote sensing image sample library, have been launched by Huawei Shengsi Community. Both were jointly created by WHU and the Huawei team. LuoJia “Gemini” is geared to serve as an open source to users all around the world, accepting performance tests and application development from the public.

Remote sensing images have gradually become an essential information resource in people’s daily life, and computers are increasingly being relied upon to accomplish tasks of classification, identification, and search regarding these images. Accomplishing this work requires the help of interpretation software.

LuoJia “Gemini”——LuoJiaSET and LuoJiaNET

LuoJiaNET is a remote sensing image intelligent interpretation software, which makes computer processing ‘smarter’ through the deep learning of artificial intelligence. Hu Xiangyun, a professor at the School of Remote Sensing and Information Engineering at WHU and also the head of the project, said, “Images are composed of pixels. One of the tasks remote sensing interpretation undertakes is to label each pixel. What is the point? Is it water, grassland, or a tree? That’s the main problem remote sensing classification deals with, the core of remote sensing interpretation.”

At present, the common method of remote sensing interpretation is to divide a large picture into pieces, analyze them, and then merge them to form another large picture. From the underlying framework design, LuoJiaNET takes the logical route of the “overall large picture”—it identifies a point and then associates it with abundant information around it. In this way, it distinguishes the pictures more accurately. “The processing of the entire large remote sensing image is directly accomplished by using the distributed computing method of operator equivalent decomposition,” said Prof. Hu, proud of this significant progress. “The interpretation accuracy has been significantly improved.” The whole process is like a painting of ‘The Vast Land’, with ravines emerging in the chest and thunder roaring beneath the pen.

Comparison of remote sensing image interpretation strategies for LuoJiaNET

LuoJiaSET is a very rich storage library of remote sensing image samples. LuoJiaNET’s specialty lies in its ability to perform deep learning and gradually improve itself in the learning process, just the way a person progresses in education through solving a lot of questions. LuoJiaSET is LuoJiaNET’s ‘question bank’. “The bigger the question bank is, the more frequently the questions are solved, and the higher the test score is.” As Prof. Hu explained, “LuoJiaSET solves the data source problem of the translation software. Only when the remote sensing image sample library is large enough and covers enough image types can it fully perform the advantages of the software.”

Compared with other foreign remote sensing image sample libraries, LuoJiaSET has a sample size of more than 5 million pieces, and it is expected to rise to more than 10 million pieces this year. As with a ‘container’, people all over the world can easily add data and content to it. LuoJiaNET is the bottom layer, and LuoJiaSET is the sample, where everyone can contribute data, models, and algorithms, thus allowing for the formation of an ecology system. We hope that in two or three years it can become the most widely used system in our remote sensing industry and be successfully spread to the world,” Prof. Hu said.

Under the care and guidance of Academician Gong Jianya of Wuhan University, LuoJiaNET brought together the WHU team of the School of Remote Sensing, the State Key Laboratory, the young and middle-aged academic backbone of the School of Computer Science, as well as the team of MindSpore framework from Huawei. The two teams shared intellectual property rights and jointly applied for invention patents.

The WHU team holding development team meetings

At present, the LuoJiaNET team consists of 12 people, with an average age of 27 years. Under the leadership of Associate Researcher Zhang Mi, in cooperation with the Huawei team, the LuoJiaNET team has developed 822 trillion codes (about 38.8 million lines) in the past one and a half years, resulting in the construction of the LuoJiaNET system. This system includes a new set of deep learning frameworks and five categories of basic remote sensing application models, namely, remote sensing scene classification, object classification, ground object classification, change detection, and multi-perspective three-dimensional reconstruction.

Prof. Zhang Mi developing LuoJiaNET

At present, LuoJiaNET and LuoJiaSET are just prototypes that require people in more fields to continue developing and enriching the image sample library. Therefore, the project team will open the software to the public. “We’ll not only have it open in the Huawei public community but also on overseas websites. We welcome everyone to work here and to help us make it better,” Prof. Hu proposed.

Just like Windows on computers and Hongmeng on mobile phones, LuoJiaNET is also a basic software. “LuoJiaNET is like the brick for buildings. Everyone can develop a variety of applications based on it, and take our bricks to build other different houses,” Prof. Hu explained, “For example, in the prevalent autonomous driving industry, on-board cameras are to conduct image recognition and analysis while on the road, and their models can also be built based on LuojiaNet. In the future, these ‘bricks’ will be built on mobile phones or satellites.”

“LuoJiaNET is the product of our independent research. It supports mainstream CPU, GPU, Windows and Linux operating systems, and optimizes Huawei’s Shengteng NPU AI software and hardware. Remote sensing is a strategic technology, and LuoJiaNET can realize autonomous control in the field of remote sensing interpretation in China,” Prof. Hu said proudly.

WHU research and development team, including Hu Xiangyun (fifth from the left), Zhang Mi (sixth from the left, in charge of LuojiaNET), and Jiang Liangcun (fourth from the left, in charge of LuoJiaSET)

With 3.88 million lines of code and a sample size of more than 5 million, LuoJia “Gemini” combines the diligent work and painstaking efforts of WHU and the Huawei team. Determined and ambitious, WHU researchers strive for self-reliance, independence and innovation to continually contribute to the world and benefit humanity.

Rewritten by Zhao Jianghan

Edited by Li Tong, Chen Jiaqi, Li Jing, Qin Shihan, Zhang Yilin, Sylvia, Xi Bingqing