Principal Investigator
Martin Clinton Tosima Manullang, Ph.D.
Assistant Professor · Informatics, ITERA
Intelligence Multimedia Research Group · Informatics, ITERA
We teach machines to see, hear, read and create.
iMul builds deep-learning and multimedia-processing methods that work with images, video, audio and text: understanding what the media contains, extracting the information hidden in it, and generating new media, often for Indonesian languages, faces and places.
Our major theme is intelligence multimedia: deep learning and multimedia processing that let machines understand media, extract information from it and generate new media.
- theses completed
- 48
- papers
- 23
- first thesis cohort
- 2018
Image & video
17 theses
Pixels and frames: recognise what is in them and measure what changes over time.
Audio & speech
5 theses
Sound and voice: classify it, transcribe it, pull apart overlapping sources.
Text & language
4 theses
Words, especially Indonesian: read the sentiment, condense it, generate it.
Multimodal
1 theses
When one medium is not enough: fuse image, sound and text into one model.
Four media. Three verbs.
UnderstandClassify, detect and recognise what the media contains.
ExtractPull out information that is not written anywhere: a count, a transcript, a heart rate.
GenerateProduce new media: enhance it, separate it, summarise it.
Image & videoAudio & speechText & languageMultimodalImage & video × Understand
Classify, detect and recognise what the media contains.
- ThesisComparing ResNet-50 and ConvNeXt-Tiny for Multi-Class Retinal Disease Classification on a Limited Dataset
- ThesisFall-Position Detection for the Elderly with a Graph Convolutional Network on Pose Landmarks
- ThesisIdentifying Cassiterite Minerals in Microscope Images with Mask R-CNN
- ThesisTransfer Learning for Face-Mask Detection in Thermal Images for Non-Contact Respiration Measurement
- ThesisAutomatic Entrance Gate for Body-Temperature Screening and Mask Compliance (Case Study: New-Normal Protocols at ITERA)
- PaperDetecting Pedestrian-Crossing Violations with a Single-Shot Detector on ESP32
- PaperAutomatic Gate for Body Temperature Check and Mask-Wearing Compliance Using an Embedded System and Deep Learning
- PaperTransformer-Based Deep Learning Model for Coffee Bean Classification
- PaperRecognition of Sign Language System for Indonesian Language Using Long Short-Term Memory Neural Networks
- PaperA Heuristic Hidden Markov Model to Recognize Inflectional Words in the Sign System for Indonesian Language (SIBI)
- PaperCombining Depth Image and Skeleton Data from Kinect for Recognizing Words in the Sign System for Indonesian Language (SIBI)
Image & video × Extract
Pull out information that is not written anywhere: a count, a transcript, a heart rate.
- ThesisReal-Time Stress Detection through HRV Analysis Using Remote Photoplethysmography (rPPG)
- ThesisPerformance of Facial Regions of Interest for Non-Contact Blood-Pressure Measurement from ICA-Based rPPG Signals and Zhou’s Method on Indonesian Faces
- ThesisComparing Regions of Interest for Non-Contact Pulse-Rate Measurement from Facial Video with the POS Algorithm
- ThesisNon-Contact Respiratory-Rate Measurement by Tracking Pixel-Intensity Changes in RGB Video
- ThesisNon-Contact Blood-Pressure Estimation from Facial Video Using Deep Learning
- ThesisNon-Contact Heart-Rate Measurement (Remote Photoplethysmography) from RGB Video
- ThesisSoftware for Non-Contact Respiratory-Rate Measurement with a Thermal Camera
- ThesisNon-Contact Respiratory Rate and Variability from Thermal Video Using Zero-Crossing and Local-Maxima Algorithms
- ThesisA Crowd-Counting Visitor-Limit Alarm for Enclosed Public Spaces
- ThesisFace-Recognition Visitor-Logging Camera for an Automatic Entrance Gate (Case Study: New-Normal Protocols at ITERA)
- PaperA Transformer-Based Network for Estimating Blood Pressure Using Facial Videos
- PaperSmart Camera for Visitor Recording Based on Face Recognition in Automatic Gates
- PaperAdapting Remote Photoplethysmography for Indonesian Subjects: An Examination of Diverse rPPG Techniques
- PaperEnhancing Visual Speech Perception through Deep Automatic Lipreading: A Systematic Review
- PaperDeveloping a Lightweight Model for Lip-Reading
Image & video × Generate
Produce new media: enhance it, separate it, summarise it.
- ThesisGenerative Networks for Digital Image Enhancement: A Super-Resolution Approach
- ThesisSmartphone Augmented Reality for Introducing the ITERA Campus
Audio & speech × Understand
Classify, detect and recognise what the media contains.
- ThesisPerformance Analysis of the PANNs Architecture Using Raw-Waveform, Log-Mel Spectrogram and Hybrid Feature Representations for Urban Emergency Sound Classification
- ThesisEnvironmental Sound Classification with Piczak’s CNN on Spectrogram Representations
Audio & speech × Extract
Pull out information that is not written anywhere: a count, a transcript, a heart rate.
- ThesisFine-Tuning OpenAI Whisper Base for Minangkabau Automatic Speech Recognition
Audio & speech × Generate
Produce new media: enhance it, separate it, summarise it.
- ThesisFine-Tuning Conv-TasNet for Choral Music Separation from a Speech-Separation Model
- ThesisImproving Conv-TasNet Speech Separation by Fine-Tuning on Low-Fidelity Indonesian Audio
Text & language × Understand
Classify, detect and recognise what the media contains.
- ThesisApplying TextCNN to Sentiment Analysis of Shopee App Reviews on the Google Play Store
- ThesisSentiment Analysis of Online-Gambling Comments on YouTube with a TextCNN Architecture
- PaperComparative Analysis of CNN, Transformers, and Traditional ML for Classifying Online Gambling Spam Comments in Indonesian
Text & language × Extract
Pull out information that is not written anywhere: a count, a transcript, a heart rate.
- ThesisFine-Tuning OpenAI Whisper Base for Minangkabau Automatic Speech Recognition
- PaperMobile-Based Question Answering System Development with Ontology-Based Knowledge
- PaperDesign of an Ontology-Based Question Answering System for the Incomplete Sentence Problem
Text & language × Generate
Produce new media: enhance it, separate it, summarise it.
- ThesisFine-Tuning a T5 Transformer for Abstractive Summarization of Indonesian Conversations
Multimodal × Understand
Classify, detect and recognise what the media contains.
- ThesisIdentifying Self-Harm Memes with an Intermediate-Fusion Multimodal Model of CLIP and ELECTRA
Multimodal × Extract
Pull out information that is not written anywhere: a count, a transcript, a heart rate.
Nothing here yet. This is a gap we would like a student or partner to take on.
Multimodal × Generate
Produce new media: enhance it, separate it, summarise it.
- PaperFrom Speech to Summary: A Pipeline-Based Evaluation of Whisper and Transformer Models for Indonesian Dialogue Summarization
Not on the map: the group’s earlier work in signals & iot (18) and data & software (5).
Publications
Show papers by
2026
- From Speech to Summary: A Pipeline-Based Evaluation of Whisper and Transformer Models for Indonesian Dialogue Summarization
- Generative AI Training for Teachers to Develop Learning Content
2025
- Comparative Analysis of CNN, Transformers, and Traditional ML for Classifying Online Gambling Spam Comments in Indonesian
- The Impact of Coloured Noise on Brainwave Activity: Implications for Indoor Environment Design
- Integrating Gradient Boosting and Parametric Architecture for Optimizing Energy Use Intensity in Net-Zero Energy Buildings
- Transformer-Based Deep Learning Model for Coffee Bean Classification
- Enhancing Visual Speech Perception through Deep Automatic Lipreading: A Systematic Review
2024
- A Transformer-Based Network for Estimating Blood Pressure Using Facial Videos
- Smart Camera for Visitor Recording Based on Face Recognition in Automatic Gates
2023
- Adapting Remote Photoplethysmography for Indonesian Subjects: An Examination of Diverse rPPG Techniques
- A Low-Cost Notion to Monitor and Diagnose Obstructive Sleep Apnea (Ideas for an Affordable Sleep Test in Indonesia)
- Developing a Lightweight Model for Lip-Reading
2022
- Detecting Pedestrian-Crossing Violations with a Single-Shot Detector on ESP32
- Automatic Gate for Body Temperature Check and Mask-Wearing Compliance Using an Embedded System and Deep Learning
2020
- IoT-Based Architecture for Automatic Detection of Fall Incidents Using Accelerometer Data
- Mobile-Based Question Answering System Development with Ontology-Based Knowledge
- An Optimization Model for Lecturer Teaching Loads that Minimizes the Mean Deviation of Teaching Load
- An Optimization Model for Teaching Assignment Based on Lecturers' Capability Using Linear Programming
2019
- Design of an Ontology-Based Question Answering System for the Incomplete Sentence Problem
- Fall Detection Algorithm Based on Accelerometer and Gyroscope Sensor Data Using Recurrent Neural Networks
2018
- Recognition of Sign Language System for Indonesian Language Using Long Short-Term Memory Neural Networks
2015
- A Heuristic Hidden Markov Model to Recognize Inflectional Words in the Sign System for Indonesian Language (SIBI)
2013
- Combining Depth Image and Skeleton Data from Kinect for Recognizing Words in the Sign System for Indonesian Language (SIBI)
Bold: iMul investigators. Underlined: iMul students. Full lists on Google Scholar: Manullang, Wisesa.
Track record
- 48Theses supervisedUndergraduate theses completed in the group since 2018.See them all
- 2Research grants
- Obstructive sleep apnea monitoring systemSimlitabmas PDP (national research grant), 2020
- Customer satisfaction cameraITERA Smart (institutional grant), 2019
- 2Registered copyrights
- Cusamera: customer-satisfaction detection from facial analysisEC00202007198, 2020
- IoT-based automatic medicine dispenser prototypeEC00202052528, 2020
- 1Textbook
- 2Alumni now teaching at IF ITERAAlya Khairunnisa Rizkita and Leslie Anggraini returned to the program as lecturers.
Members
Co-Principal Investigator
I Wayan Wiprayoga Wisesa
Lip-reading · Sign-language recognition · Affective computing
Students, in two subgroups
Pixel LabImage & video
Faces, video and medical images, with models that explain themselves
- Sigit Kurnia Hartawan123140033Explainable AI to reveal which facial regions drive blood-pressure estimation
- Elfa Noviana Sari123140066Demographic bias and fairness of deep-learning remote photoplethysmography (rPPG) across skin tones
- Memory Simanjuntak123140095Cross-population validation of computer-vision models that detect student engagement (DAiSEE dataset)
- Romualdus Hary Prabowo123140083Lightweight CNN compression and the reliability of explainable AI (XAI): faithfulness and consistency on medical images
- Juliani Leony Putri Melati Manalu123140029Quantitative faithfulness of explainable-AI methods on Vision Transformers for diabetic retinopathy grading
- Muhammad Fadhilah Akbar123140003BISINDO sign-language alphabet classification with hybrid multi-variant MobileViT and weighted cross-entropy loss
WaveformSignals
Brain and heart signals (EEG, ECG) read by interpretable models
- Riyan Sandi Prayoga123140176Multilabel ECG classification with unidirectional Mamba
- Mohd. Musyaffa Alief Athallah123140184Interpretable machine learning with XGBoost and SHAP to classify thermal comfort from EEG signals (HERO dataset)
- Muhammad Fajri Firdaus123140050Interpretable machine learning to find shared and disorder-specific qEEG markers across major psychiatric disorders
48 theses and counting
Every thesis finished in the group, from early campus IoT prototypes to deep learning on images, sound and text.
Filter by medium
48 theses
2026
- Performance Analysis of the PANNs Architecture Using Raw-Waveform, Log-Mel Spectrogram and Hybrid Feature Representations for Urban Emergency Sound Classification
- Drowsiness Level Detection from EEG and EOG Signals Using the Mamba Model: A Comparison of Regression and Classification
- Comparing XGBoost and GRU for Sea-Surface Temperature Prediction from BMKG Model Data in Lampung Bay
- A Machine-Learning Regression Model for Estimating Lecturer Stress Scores Based on the Short Stress Overload Scale (SOS-S)
- Comparing ResNet-50 and ConvNeXt-Tiny for Multi-Class Retinal Disease Classification on a Limited Dataset
- Identifying Self-Harm Memes with an Intermediate-Fusion Multimodal Model of CLIP and ELECTRA
- Fine-Tuning OpenAI Whisper Base for Minangkabau Automatic Speech Recognition
- Fine-Tuning a T5 Transformer for Abstractive Summarization of Indonesian Conversations
- Predicting English Premier League Match Outcomes from Performance Statistics and Elo Ratings with TabPFN-2.5
- Fine-Tuning Conv-TasNet for Choral Music Separation from a Speech-Separation Model
- Environmental Sound Classification with Piczak’s CNN on Spectrogram Representations
- Comparing Encoding Techniques in Machine Learning for Heart-Attack Prediction in Indonesia
- Improving Conv-TasNet Speech Separation by Fine-Tuning on Low-Fidelity Indonesian Audio
- Fall-Position Detection for the Elderly with a Graph Convolutional Network on Pose Landmarks
- Real-Time Stress Detection through HRV Analysis Using Remote Photoplethysmography (rPPG)
- Identifying Cassiterite Minerals in Microscope Images with Mask R-CNN
- Performance of Facial Regions of Interest for Non-Contact Blood-Pressure Measurement from ICA-Based rPPG Signals and Zhou’s Method on Indonesian Faces
- Applying TextCNN to Sentiment Analysis of Shopee App Reviews on the Google Play Store
- Sentiment Analysis of Online-Gambling Comments on YouTube with a TextCNN Architecture
- A Web-Based Point-of-Sale System Developed with the Modified Waterfall Method (Case Study: Toko Merapi)
2018–2025
- Comparing Regions of Interest for Non-Contact Pulse-Rate Measurement from Facial Video with the POS Algorithm
- Non-Contact Respiratory-Rate Measurement by Tracking Pixel-Intensity Changes in RGB Video
- Non-Contact Blood-Pressure Estimation from Facial Video Using Deep Learning
- Generative Networks for Digital Image Enhancement: A Super-Resolution Approach
- Non-Contact Heart-Rate Measurement (Remote Photoplethysmography) from RGB Video
- Software for Non-Contact Respiratory-Rate Measurement with a Thermal Camera
- Non-Contact Respiratory Rate and Variability from Thermal Video Using Zero-Crossing and Local-Maxima Algorithms
- Transfer Learning for Face-Mask Detection in Thermal Images for Non-Contact Respiration Measurement
- Automatic Hydroponic Nutrient Control Using an Artificial Neural Network
- A Physical-Distancing Monitoring Device Based on IoT and Radio Frequency
- Diagnosing Obstructive Sleep Apnea from Sleep Activity
- A Crowd-Counting Visitor-Limit Alarm for Enclosed Public Spaces
- Prototype of an Automatic Seedling Incubator for Solanaceae Plants Using Arduino Uno
- An ESP32-Based IoT Baby Swing for Remote Infant Monitoring
- Monitoring and SMS Alerts for Soil Moisture and Air Pressure at Fire-Prone Sites
- IoT-Based Tipping-Bucket Rainfall Monitoring Prototype (Case Study: Margatiga Dam)
- Smart-Room Prototype with Biometric Door Security (Case Study: ITERA Facilities Office)
- Prototype of an RFID-Based Locker Storage System
- Flood-Prediction Monitoring from Water Discharge and Level Using ESP8266 and Fuzzy Logic
- Automatic Entrance Gate for Body-Temperature Screening and Mask Compliance (Case Study: New-Normal Protocols at ITERA)
- Face-Recognition Visitor-Logging Camera for an Automatic Entrance Gate (Case Study: New-Normal Protocols at ITERA)
- Forest-Fire Monitoring System Based on a Wireless Sensor Network
- Smartphone Augmented Reality for Introducing the ITERA Campus
- IoT-Based Automatic Medicine Dispenser Prototype for Medication Reminders
- Heart-Rhythm Monitoring with Web and Mobile Interfaces for Arrhythmia Detection
- Automatic Plant-Watering Prototype Using MQTT and Sugeno Fuzzy Logic (Case Study: ITERA)
- Prototype of an IoT-Enabled Automatic Trash Bin
- Real-Time Atrial Fibrillation Detection Using R-R Interval Filtering in a Python GUI