On Distributed Deep Network for Processing Large-Scale Sets of Complex Data

Conference paper

Chen, D (2016). On Distributed Deep Network for Processing Large-Scale Sets of Complex Data. 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). Hangzhou, China 27 - 28 Aug 2016
AuthorsChen, D
TypeConference paper
Abstract

Recent work in unsupervised feature learning and deep learning has shown that being able to train large models can dramatically improve performance. In this paper, we consider the problem of training a deep network with hundreds of parameters using distributed CPU cores. We have developed Bagging-Down SGD algorithm to solve the distributing problems. Bagging-Down SGD introduces the parameter server adding on the several model replicas, and separates the updating and the training computing to accelerate the whole system. We have successfully used our system to train a distributed deep network, and achieve state-of-the-art performance on MINIST, a visual handwriting font library. We show that these techniques dramatically accelerate the training of this kind of distributed deep network.

Year2016
Accepted author manuscript
License
CC BY 4.0
File Access Level
Open
Publication dates
Print27 Aug 2016
Publication process dates
Deposited12 Aug 2016
Accepted27 Jul 2016
Permalink -

https://openresearch.lsbu.ac.uk/item/872q3

Download files

Accepted author manuscript
PID4291337.pdf
License: CC BY 4.0
File access level: Open

  • 20
    total views
  • 3
    total downloads
  • 0
    views this month
  • 3
    downloads this month

Export as

Related outputs

An Adaptive Task Scheduling Method for Networked UAV Combat Cloud System Based on Virtual Machine and Task Migration
Li, B., Liang, S., Tian, L., Chen, D. and Zhang, M. (2020). An Adaptive Task Scheduling Method for Networked UAV Combat Cloud System Based on Virtual Machine and Task Migration. Mathematical Problems in Engineering. p. 5391479.
An adaptive dwell time scheduling model for phased array radar based on three-way decision
Li, B., Tian, L., Chen, D. and Liang, S. (2020). An adaptive dwell time scheduling model for phased array radar based on three-way decision. Journal of Systems Engineering and Electronics.
Intelligent aircraft maneuvering decision based on CNN
Li, B, Liang, S, Tian, L and Chen, D (2019). Intelligent aircraft maneuvering decision based on CNN. the 3rd International Conference on Computer Science and Application Engineering. Sanya, China 22 - 24 Oct 2019 ACM Press. doi:10.1145/3331453.3362046
Intelligent Attitude Control of Aircraft Based on LSTM
Li, B, Gao, P, Li, X and Chen, D (2019). Intelligent Attitude Control of Aircraft Based on LSTM. 3rd International Conference on Artificial Intelligence Applications and Technologies. Beijing, China 01 - 03 Aug 2019 IOP Publishing. doi:10.1088/1757-899X/646/1/012013
A Task Scheduling Algorithm for Phased Array Radar Based on Dynamic Three-way Decision
Li, B., Tian, L., Chen, D. and Han, Y. (2019). A Task Scheduling Algorithm for Phased Array Radar Based on Dynamic Three-way Decision. Sensors. 20 (1).
Intelligent Flight Control of Combat Aircraft Based on Autoencoder
Li, B., Gao, P., Liang, S. and Chen, D. (2019). Intelligent Flight Control of Combat Aircraft Based on Autoencoder. 2019 The 4th International Conference on Robotics, Control and Automation. GuangZhou 26 - 28 Jul 2019 doi:10.1145/3351180.3351210
Skin Capacitive Imaging Analysis Using Deep Learning GoogLeNet
Zhang, X., Pan, W., Bontozoglou, C., Chirikhina, E., Chen, D. and Xiao, P. (2019). Skin Capacitive Imaging Analysis Using Deep Learning GoogLeNet. Computing Conference 2020. London, UK 16 - 17 Jul 2019 Springer.
FRS: A Simple Knowledge Graph Embedding Model for Entity Prediction
Wang, L.F., Lu, X., Jiang, Z., Zhang, Z., Li, R., Zhao, M. and Chen, D. (2019). FRS: A Simple Knowledge Graph Embedding Model for Entity Prediction. Mathematical Biosciences and Engineering. 16 (6), pp. 7789-7807.
Predicting Customer Profitability Dynamically over Time: An Experimental Comparative Study
Chen, D., Guo, K. and Li, B. (2019). Predicting Customer Profitability Dynamically over Time: An Experimental Comparative Study. 24th Iberoamerican Congress on Pattern Recognition (CIARP 2019). Havana, Cuba 28 - 31 Oct 2019 doi:10.1007/978-3-030-33904-3_16
Learning Bayesian Networks using the Constrained Maximum a Posteriori Probability Method
Yang, Y, Gao, X, Guo, Z and Chen, D (2019). Learning Bayesian Networks using the Constrained Maximum a Posteriori Probability Method. Pattern Recognition. 91, pp. 123-134.
Learning Bayesian network parameters via minimax algorithm
Gao, X, Gao, G, Ren, H, Chen, D and He, C (2019). Learning Bayesian network parameters via minimax algorithm. International Journal of Approximate Reasoning. 108, pp. 62-75.
Improving Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnosis via RBF Networks trained with EKF models
Adegoke, V, Chen, D and Banissi, E (2019). Improving Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnosis via RBF Networks trained with EKF models. International Journal of Computer Information Systems and Industrial Management.
A New Supervised t-SNE with Dissimilarity Measure for Effective Data Visualization and Classification
Hajderanj, L, Weheliye, I and Chen, D (2019). A New Supervised t-SNE with Dissimilarity Measure for Effective Data Visualization and Classification. 2019 8th International Conference on Software and Information Engineering. Cairo 09 - 12 Apr 2019
Recurrent Neural Networks for Decoding Lip Read Speech
Fenghour, S, Chen, D and Xiao, P (2019). Recurrent Neural Networks for Decoding Lip Read Speech. 2019 8th International Conference on Software and Information Engineering (ICSIE 2019). Cairo 09 - 12 Apr 2019
Decoder-Encoder LSTM for Lip Reading
Fanghour, S, Chen, D and Xiao, P (2019). Decoder-Encoder LSTM for Lip Reading. 2019 8th International Conference on Software and Information Engineering (ICSIE 2019). Cairo, Eygpt 09 - 12 Apr 2019
Enhancing Ensemble Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnostic using optimized EKF-RBFN trained prototypes, The 10th International Conference on Soft Computing and Pattern Recognition
Adegoke, V, Chen, D, Banissi, E and Barikzai, S (2019). Enhancing Ensemble Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnostic using optimized EKF-RBFN trained prototypes, The 10th International Conference on Soft Computing and Pattern Recognition. The 10th International Conference on Soft Computing and Pattern Recognition. Porto, Portugal 13 - 15 Dec 2018
Distributed deep networks based on Bagging-Down SGD algorithm
Qin, C, Gao, X and Chen, D (2019). Distributed deep networks based on Bagging-Down SGD algorithm. Xi Tong Gong Cheng Yu Dian Zi Ji Shu/Systems Engineering and Electronics. 41 (5), pp. 1021-1027.
Towards automated cost analysis, benchmarking and estimating in construction: a machine learning approach
Chen, D, Hajderanj, L and Fiske, J (2019). Towards automated cost analysis, benchmarking and estimating in construction: a machine learning approach. 13th Multi Conference on Computer Science and Information Systems (MCCSIS). Porto, Portugal 16 - 18 Jul 2019
Design of a voice control 6DoF grasping robotic arm based on ultrasonic sensor, computer vision and Alexa voice assistance
Wang, Z, Chen, D and Xiao, P (2019). Design of a voice control 6DoF grasping robotic arm based on ultrasonic sensor, computer vision and Alexa voice assistance. International Conference on Information Technology in Medicine and Education. Qingdao, China 23 - 25 Aug 2019 IEEE. doi:10.1109/ITME.2019.00150
Visual analytics in the public sector: An analysis on diversities and similarities of London’s wards
Chen, D, Sanz, BM and Zhao, E (2018). Visual analytics in the public sector: An analysis on diversities and similarities of London’s wards. International Conference on Big Data Analytics, Data Mining and Computational Intelligence 2018 (BigDaCI 2018). Madrid, Spain 18 - 20 Jul 2018 Bigdaci.
Contour Mapping for Speaker-Independent Lip Reading System
Fenghour, S, Chen, D and Xiao, P (2018). Contour Mapping for Speaker-Independent Lip Reading System. The 11th International Conference on Machine Vision (ICMV 2018). Munich, Germany 01 - 03 Nov 2018
Determining Key (Predictor) Modules for Early Identification of Students At-Risk
Chen, D and Elliott, G (2013). Determining Key (Predictor) Modules for Early Identification of Students At-Risk. International Conference on Advanced Information Engineering and Education Science (ICAIEES 2013). Beijing, China 19 - 20 Dec 2013 London South Bank University. doi:10.2991/icaiees-13.2013.22
Big Data Analytics System for Fact/Data-driven Decision Making
Chen, D (2015). Big Data Analytics System for Fact/Data-driven Decision Making. The Royal Statistical Society, Business and Industry Section. London, UK 18 Nov 2015 London South Bank University.
Learning Bayesian Network Parameters from a Small Data Set: A Further Constrained Qualitatively Maximum a Posteriori Method
Guo, Zhi-gao, Gao, Xiao-guang, Hao, Ren, Yang, Yu, Di, Ruo-hai and Chen, D (2017). Learning Bayesian Network Parameters from a Small Data Set: A Further Constrained Qualitatively Maximum a Posteriori Method. International Journal of Approximate Reasoning. 91 (Dec), pp. 22-35.
Feature Extraction and Labelling Large Data Sets Using Deep Learning
Chen, D (2017). Feature Extraction and Labelling Large Data Sets Using Deep Learning. RESEARCHER LINK: Smart Technology for Fighting Virus Epidemics & Bioinformatics. Recife, Pernambuco, Brazil 10 - 13 Sep 2017
Prediction of Breast Cancer Survivability using Ensemble Algorithms
Adegoke, V, Chen, D, Banissi, E and Barikzai, S (2017). Prediction of Breast Cancer Survivability using Ensemble Algorithms. International Conference on Smart System and Technologies 2017 (SST 2017),. Osijek, Croatia 18 - 20 Oct 2017
Predictive Ensemble Modelling: An Experimental Comparison of Boosting Implementation Methods
Adegoke, V, Chen, D, Barikzai, S and Banissi, E (2017). Predictive Ensemble Modelling: An Experimental Comparison of Boosting Implementation Methods. 2017 European Modelling Symposium (EMS). Manchester 20 - 21 Nov 2017
Making Better Use of Big Data
Chen, D (2016). Making Better Use of Big Data. LSBU Enterprise Count Event, March 2016. London Southbank University 18 - 18 Mar 2016 London South Bank University.
Big Data Analytics In The Public Sector: A Case Study Of NEET Analysis For The London Boroughs
Chen, D, Asaolu, B and Qin, C (2016). Big Data Analytics In The Public Sector: A Case Study Of NEET Analysis For The London Boroughs. International Conference on Big Data Analytics, Data Mining and Computational Intelligence. Funchal, Madeira, Portugal 02 - 04 Jul 2016
On Distributed Deep Network for Processing Large-Scale Sets of Complex Data
Qin, C, Gao, X and Chen, D (2016). On Distributed Deep Network for Processing Large-Scale Sets of Complex Data. 2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). Hangzhou, China. 27 - 28 Aug 2016 Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/IHMSC.2016.55
A Bayesian Approach to Learn Bayesian Networks Using Data and Constraints
Gao, X, Yu, Y, Zhi-gao, G and Chen, D (2016). A Bayesian Approach to Learn Bayesian Networks Using Data and Constraints. 23rd International Conference on Pattern Recognition (ICPR 2016). Cancún, México 04 - 08 Dec 2016 Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/ICPR.2016.7900204
Data mining for the online retail industry: A case study of RFM model-based customer segmentation using data mining
Chen, D (2012). Data mining for the online retail industry: A case study of RFM model-based customer segmentation using data mining. Journal of Database Marketing and Customer Strategy Management. 19 (3), pp. 197-208.