- Proposed a scheduling-aware data prefetching mechanism. By evaluation, the mechanism reduced 14% and 17% Spark execution time with 25 GB and 50 GB datasets.
- The prefetch mechanism was based on Spark task scheduling, data was prefetched to cache just before it will be processed.
- The cache system was based on Alluxio, an open source in-memory distributed storage system.
- Accepted by The 31st IEEE International Conference on Advanced Information Networking and Applications (AINA-2017), [ pdf ].
- Accepted by IEEE Transactions on Parallel and Distributed Systems.
- A Convolution Neural Network (CNN) based neural network image classification system.
- Used Keras and TensorFlow to train the classification model; used Django and Python to develop the classifier website. Demo classifier website was deployed on Microsoft Azure.
Taipei Weather Bot