Based on the ability of the Ant Group and Intel BigDL team to run the Apache Spark distributed Big data processing application in Occlum
Dec 12, 2022| Based on the ability of the Ant Group and Intel BigDL team to run the Apache Spark distributed Big data processing application in Occlum, Ant Group and Intel teams jointly released "Build End-To-End Secure Distributed Spark Big Data Analysis Solution Based on Occlum and BigDL", which also came online in release 1.0 of Occlum.
Compared with existing Spark on SGX solutions in the industry, this solution not only supports multiple deployment modes but also makes up for the shortcomings of existing Spark solutions in terms of capability support.
Based on the secure operating environment provided by Occlum, a distributed privacy protection machine learning platform (PPML) is built based on BigDL, which can protect end-to-end distributed AI applications (including data input, data analysis, machine learning, deep learning, and other phases). Unlike traditional privacy computing frameworks, the solution provides an environment in which standard big data applications can be run. The hope is to help existing big data/distributed applications seamlessly migrate to an end-to-end secure environment and strengthen security at every step.



