Scheduling-Aware Data Prefetching Based on Spark Framework
- Proposed a scheduling-aware data prefetching mechanism. By evaluation, the mechanism reduced 14% and 17% Spark execution time with 25 GB and 50 GB datasets.
- The prefetch mechanism was based on Spark task scheduling, data was prefetched to cache just before it will be processed.
- The cache system was based on Alluxio, an open source in-memory distributed storage system.
- Accepted by The 31st IEEE International Conference on Advanced Information Networking and Applications (AINA-2017), [ pdf ].
- Accepted by IEEE Transactions on Parallel and Distributed Systems.
FileBox
Car Classifier
- A Convolution Neural Network (CNN) based neural network image classification system.
- Used Keras and TensorFlow to train the classification model; used Django and Python to develop the classifier website. Demo classifier website was deployed on Microsoft Azure.
- Demo
- ly2314/NotHotdog-Classifier
- ly2314/CarIdentifier-Web
PLURKLY!
Daily Pixiv
- A Pixiv daily ranking viewer on Windows Phone.
- Daily Pixiv is available on Microsoft Store.
Taipei Weather Bot
- 台北天氣小幫手: A weather reporting bot on Plurk.
- ly2314/PlurkBot-TaipeiWeather
Painter
- A painter like application build by Qt.
- ly2314/NMLab_Painter
- ly2314/QtPainter_Qt5