Machine Learning for Automated Quality Evaluation in Pharmaceutical Manufacturing of Emulsions

  • Saritha UnnikrishnanEmail author
  • John Donovan
  • Russell Macpherson
  • David Tormey
Original Article



In pharmaceutical industries, the quality assessment of emulsions is typically based on subjective examination of these samples under the microscope by trained analysts. The major drawbacks of such manual quality assessment include inter-observer variability, intra-observer variability, lack of speed and poor accuracy. In order to address these challenges, an automated approach, based on machine vision and machine learning, is investigated in this study.


Micrographs, obtained during an emulsification process, are classified into four quality-based categories named TAMU (target, acceptable, marginal and unacceptable). A machine learning approach using principal component–based discriminant analysis is employed in this study for the classification. This approach is compared with manual classification results obtained for the same set of micrographs using attribute agreement analysis, which is a methodology of assessing the accuracy and precision of an evaluation system.


The automated approach is demonstrated to be repeatable, 40% more accurate compared to the least performing analyst and 10% more accurate than the best performing analyst. The results show that the automated classification is superior to manual classification of micrographs with respect to speed (180 times faster), greater accuracy and repeatability.


The automated approach, implemented as a soft sensor, integrated with real-time image acquisition can be applied for in situ process monitoring of emulsions. The real-time approach can be used to predict the instantaneous product quality as well as optimum process time required to achieve the desirable droplet characteristics, which will avoid over-processing and wastage of resources in pharmaceutical industries.


Machine learning Machine vision Automated quality evaluation Emulsion processing Manual assessment Attribute agreement analysis 



The authors wish to thank Dr. Stephen Finn (GlaxoSmithKline, Sligo, Ireland) for providing the microscopic images for analysis.

Funding Information

This work was financially supported by Institute of Technology Sligo’s President’s bursary award. The North West Centre for Advanced Manufacturing (NW CAM) project is supported by the European Union’s INTERREG VA Programme, managed by the Special EU Programmes Body (SEUPB).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.


The views and opinions in this document do not necessarily reflect those of the European Commission or the Special EU Programmes Body (SEUPB).


  1. 1.
    Anil M, Govindaraju M, Subramani B. A comparison between manual and hybrid methods in parts inspection. Integr Manuf Syst. 1998;9(6):344–9. Scholar
  2. 2.
    Malamas EN, Petrakis EGM, Zervakis M, Petit L, Legat J-D. A survey on industrial vision systems, applications and tools. Image Vis Comput. 2003;21(2):171–88. Scholar
  3. 3.
    Marques C, Lopes N, Santos G, Delgado I, Delgado P. Improving operator evaluation skills for defect classification using training strategy supported by attribute agreement analysis. Measurement. 2018;119:129–41. Scholar
  4. 4.
    Laofor C, Peansupap V. Defect detection and quantification system to support subjective visual quality inspection via a digital image processing: a tiling work case study. Autom Constr. 2012;24:160–74. Scholar
  5. 5.
    Wesley R, Sankaranarayanan R, Mathew B, Chandralekha B, Aysha Beegum A, Amma NS, et al. Evaluation of visual inspection as a screening test for cervical cancer. Br J Cancer. 1997;75:436–40. Scholar
  6. 6.
    Chiu M-C, Yeh L-J, Hsu C-J. The application of image division method on automatic optical inspection of PCBA. J Inf Optim Sci. 2010;31(2):257–74. Scholar
  7. 7.
    Brosnan T, Sun D-W. Improving quality inspection of food products by computer vision–a review. J Food Eng. 2004;61(1):3–16. Scholar
  8. 8.
    Gosselin R, Vachon Lachance E, Cournoyer A, Clarke F. Classifying pharmaceutical capsules through X-ray image analysis based on the agglomeration of their contents. J Pharm Innov. 2016;11(1):92–101. Scholar
  9. 9.
    Islam MJ, Basalamah SM, Ahmadi M, Sid-Ahmed MA. Computer vision-based quality inspection system of transparent gelatin capsules in pharmaceutical applications. Am J Intell Syst. 2012;2(1):14–22.Google Scholar
  10. 10.
    Al-Refaie A, Bata N. Evaluating measurement and process capabilities by GR&R with four quality measures. Measurement. 2010;43(6):842–51. Scholar
  11. 11.
    Browne R, MacKay J, Steiner S. Leveraged gauge R&R studies. Technometrics. 2010;52(3):294–302. Scholar
  12. 12.
    Murphy SA, Moeller SE, Page JR, Cerqua J, Boarman M. Leveraging measurement system analysis (MSA) to improve library assessment: the attribute gage R&R. Coll Res Libr. 2009;70(6):568–77. Scholar
  13. 13.
    Vago E, Kemeny S. Random effects model for attribute gauge R&R. Qual Reliab Eng Int. 2012;28(8):807–23. Scholar
  14. 14.
    Lyu J, Chen M-N. Gauge capability studies for attribute data. Qual Reliab Eng Int. 2008;24(1):71–82. Scholar
  15. 15.
    Chen M, Lyu J. Enhancement of measurement capability for precision manufacturing processes using an attribute gauge system. Proc Inst Mech Eng B J Eng Manuf. 2011;225(B10):1912–24. Scholar
  16. 16.
    de Mast J, van Wieringen WN. Modeling and evaluating repeatability and reproducibility of ordinal classifications. Technometrics. 2010;52(1):94–106. Scholar
  17. 17.
    Minitab, Attribute Agreement Analysis, n.d. http:// Accessed 02/05/2018.
  18. 18.
    Ulery B, Hicklin R, Buscaglia J, Roberts M. Repeatability and reproducibility of decisions by latent fingerprint examiners. PLoS One. 2012;7(3).
  19. 19.
    Rigon A, Infantino M, Merone M, Lannello G, Tincani A, Cavazzana I, et al. The inter-observer reading variability in anti-nuclear antibodies indirect (ANA) immunofluorescence test: a multicenter evaluation and a review of the literature. Autoimmun Rev. 2017;16(12):1224–9. Scholar
  20. 20.
    Gavrielides MA, Gallas BD, Lenz P, Badano A, Hewitt SM. Observer variability in the interpretation of HER2/neu immunohistochemical expression with unaided and computer-aided digital microscopy. Arch Pathol Lab Med. 2011;135(2):233–42. Scholar
  21. 21.
    Chung K, Crane MM, Lu H. Automated on-chip rapid microscopy, phenotyping and sorting of C. elegans. Nat Methods. 2008;5:637–43. (Accessed: 22 June 2018).
  22. 22.
    Gosnell ME, Polikarpov DM, Goldys EM, Zvyagin AV, Gillatt DA. Computer-assisted cystoscopy diagnosis of bladder cancer. Urol Oncol. 2018;36(1):7–8.e15. Scholar
  23. 23.
    Meijer N, Abbes H, Hansen WG. Particle size distribution and dispersion of oil-in-water emulsions: an application of light microscopy. Am Lab. 2001;33(8):28–+.Google Scholar
  24. 24.
    Junker B. Measurement of bubble and pellet size distributions: past and current image analysis technology. Bioprocess Biosyst Eng. 2006;29(3):185–206. Scholar
  25. 25.
    Gwyn JE, Crosby EJ, Marshall WR. Bias in particle-size analyses by count method. Ind Eng Chem Fundam. 1965;4(2):204–8. Scholar
  26. 26.
    Kljusuric J, Benkovic M, Bauman I. Classification and processing optimization of barley milk production using NIR spectroscopy, particle size, and total dissolved solids analysis. J Chem. 2015;2015:1–7. Scholar
  27. 27.
    Sharma S, Dhalsamant K, Tripathy PP. Application of computer vision technique for physical quality monitoring of turmeric slices during direct solar drying. J Food Meas Charact. 2018;13:545–58. Scholar
  28. 28.
    Zeaiter J, Romagnoli JA, Gomes VG. Online control of molar mass and particle-size distributions in emulsion polymerization. AICHE J. 2006;52(5):1770–9. Scholar
  29. 29.
    Hu YS, Wang ZJ, Fan XG, Li JJ, Gao A. Material microstructures analyzed by using gray level co-occurrence matrices. Chin Phys B. 2017;26(9):8. Scholar
  30. 30.
    Boxall JA, Koh CA, Sloan ED, Sum AK, Wu DT. Measurement and calibration of droplet size distributions in water-in-oil emulsions by particle video microscope and a focused beam reflectance method. Ind Eng Chem Res. 2010;49(3):1412–8. Scholar
  31. 31.
    Khalil A, Puel F, Chevalier Y, Galvan JM, Rivoire A, Klein JP. Study of droplet size distribution during an emulsification process using in situ video probe coupled with an automatic image analysis. Chem Eng J. 2010;165(3):946–57. Scholar
  32. 32.
    Maaß S, Rojahn J, Haensch R, Kraume M. Automated drop detection using image analysis for online particle size monitoring in multiphase systems. Comput Chem Eng. 2012;45:27–37. Scholar
  33. 33.
    Scherze I, Knofel R, Muschiolik G. Automated image analysis as a control tool for multiple emulsions. Food Hydrocoll. 2005;19(3):617–24. Scholar
  34. 34.
    Freire MG, Dias AMA, Coelho MAZ, Coutinho JAP, Marrucho IM. Aging mechanisms of perfluorocarbon emulsions using image analysis. J Colloid Interface Sci. 2005;286(1):224–32. Scholar
  35. 35.
    Cardona J, Ferreira C, McGinty J, Hamilton A, Agimelen OS, Cleary A, et al. Image analysis framework with focus evaluation for in situ characterisation of particle size and shape attributes. Chem Eng Sci. 2018;191:208–31. Scholar
  36. 36.
    Schorsch S, Ochsenbein DR, Vetter T, Morari M, Mazzotti M. High accuracy online measurement of multidimensional particle size distributions during crystallization. Chem Eng Sci. 2014;105:155–68. Scholar
  37. 37.
    Schindelin J, editor. Fiji is just ImageJ (batteries included). ImageJ user and developer conference 2008.Google Scholar
  38. 38.
    Schindelin J, Arganda-Carreras I, Frise E, Kaynig V, Longair M, Pietzsch T, et al. Fiji: an open-source platform for biological-image analysis. Nat Meth. 2012;9(7):676–82 doi: (Accessed: 22 June 2018).
  39. 39.
    Schindelin J, Rueden CT, Hiner MC, Eliceiri KW. The ImageJ ecosystem: an open platform for biomedical image analysis. Mol Reprod Dev. 2015;82(7–8):518–29. Scholar
  40. 40.
    Schneider CA, Rasband WS, Eliceiri KW. NIH image to ImageJ: 25 years of image analysis. Nat Methods. 2012;9:671–5. Scholar
  41. 41.
    R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing 2017.Google Scholar
  42. 42.
    Venora G, Grillo O, Saccone R. Quality assessment of durum wheat storage centres in Sicily: evaluation of vitreous, starchy and shrunken kernels using an image analysis system. J Cereal Sci. 2009;49(3):429–40. Scholar
  43. 43.
    Peres-Neto P, Jackson D, Somers K. How many principal components? Stopping rules for determining the number of non-trivial axes revisited. Comput Stat Data Anal. 2005;49(4):974–97. Scholar
  44. 44.
    Wold S, Kim E, Geladi P. Principal component analysis. Chemom Intell Lab Syst. 1987;2(1–3):37–52. Scholar
  45. 45.
    Jackson JE. Components and factor analysis: part I - principal components. J Qual Technol. 1980;12(4):201–13.Google Scholar
  46. 46.
    Jackson JE. A user’s guide to principal components. Hoboken, New Jersey & Canada: John Wiley & Sons; 2003.Google Scholar
  47. 47.
  48. 48.
    Pfeil J, Frohme M, Schulze K. Mobile microscopy and automated image analysis: the ease of cell counting and classification. Optik Photonik. 2018;13(1):36–9.Google Scholar
  49. 49.
    Schulze K, Tillich UM, Dandekar T, Frohme M. PlanktoVision - an automated analysis system for the identification of phytoplankton. BMC Bioinf. 2013;14(1):115. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Faculty of EngineeringInstitute of Technology SligoSligoIreland
  2. 2.Centre for Precision Engineering, Materials and Manufacturing Research (PEM)Institute of Technology SligoSligoIreland
  3. 3.GlaxoSmithKlineSligoIreland

Personalised recommendations