Image & Acoustic Signals Analysis

See, Hear, Understand—Teaching Machines to Perceive the World Like We Humans Do

Image & Acoustic Signals Analysis (IASA) uses state-of-the-art equipment and techniques to analyze digitized signals, for example, for facial and speech recognition. The research carried out by first-year students in the IASA research stream at Binghamton University addresses a variety of problems ranging from autism classification through gaze, automatic sign language recognition through gesture recognition, and facial expression recognition through state of the art techniques.

Venn diagram illustrating the relationship between AI fields. A large outer ellipse labeled 'Artificial Intelligence' contains three overlapping circles: 'Computer Vision' on the left (purple), 'Signal Analysis' on the right (teal), and 'Machine Learning' in the center (green). At the core where all areas converge is a red circle labeled 'IASA,' indicating it draws from machine learning, computer vision, and signal analysis within the broader AI domain.
IASA sits at the intersection of Computer Vision, Signal Analysis, and Machine Learning—combining techniques from all three disciplines within the broader field of Artificial Intelligence.

IASA research intersects the traditional disciplines of Computer Science, Computer Engineering and Electrical Engineering. FRI IASA students tackle state of the art questions related to biometrics, human-computer interaction and robotics. The questions answered here will help give people a better quality of life, increase communication and make the world a safer place to live.

Research Themes

Privacy & Anonymization
Deep Fakes
AI for Good
Biological Signal Analysis
Synthetic Media & Image Generation
Facial Expression Recognition

Research Educator


Dr. Umur Ciftci has been the Research Educator for Image and Acoustic Signals Analysis research stream since Spring 2019. He received his PhD in computer science from Binghamton University in 2021 where he was part of the Graphics and Image Computing Laboratory. His research interests are in computer vision, human-computer interaction and affective computing.

Research Techniques

Core Skills on the left and Tools on the right, both with dark green headers. The Core Skills section contains three categories stacked vertically, each with an accompanying image: (1) Computer Vision (with an eye icon overlaid on digital imagery) - includes Image Analysis, Video Analysis, and Object Recognition. (2) Artificial Intelligence (with an illustration of a glowing human head profile with neural network patterns representing a brain) - includes Deep Learning, Generative AI, and AI Agents. (3) Human-Computer Interaction (with a photo of a woman wearing a virtual reality headset) - includes Multimodal Interfaces, AI for Good, and Biofeedback Systems. The Tools section on the right lists three technologies with their logos: (1) Programming - Python (shown with the blue and yellow Python logo), (2) Deep Learning - PyTorch (shown with the orange PyTorch flame logo) (3) Computer Vision - OpenCV (shown with the red, green, and blue OpenCV logo).The overall design uses a teal/dark green color scheme with white text and rounded rectangular containers.
Students in the Image & Acoustic Signals Analysis stream develop expertise in Computer Vision, Artificial Intelligence, and Human-Computer Interaction, while gaining hands-on experience with industry-standard tools including Python, PyTorch, and OpenCV.

Research Projects

  • Cohort 11 (2024-2025)
    • AuraCity: Multi-agent system for complex task decomposition
    • Dino-tective: Using DinoV3 to detect AI generated images
    • MultEmotion: Late temporal fusion of EEG, facial expression, and heart rate for multimodal emotion recognition
    • YANA: Your ASL native assistant
  • Cohort 10 (2023-2024)
    • Automated vehicle identification using traffic infrastructure
    • Built DIFFerent: changing the seasons of images using diffusion
    • Fantastic fashion: a blend of technology and style
    •  D³: a dynamic deepfake dataset 
    •  Synthesized satellite imagery: how fine tuning improves diffusion model quality
  • Cohort 9 (2022-2023)
    •  Automatic strike zone detection in baseball 
    • Black-box adversarial face transformation network
    • Investigation of racial bias in facial recognition algorithms
    • Unveiling the digital masquerade: Techniques in deepfake detection & sourcing
  • Cohort 8 (2021-2022)
    • CopyMarth: replicating player behavior in Super Smash Brothers Melee
    • Generating NeRF-based high fidelity head portraits
    • Text-to-image building facade synthesis
    • Transfering 2D garments onto 3D models using a clothing-segmentation
    • Utilizing deepfakes to anonymize children online
  • Cohort 7 (2020-2021)
    • Auto-generating commentary for esports
    • Deepfake video detection by analyzing facial landmark locations
    • Evaluating the impact of external stimuli in human-robot interaction
    • Integrating deep q-learning with learning from demonstration to solve Atari games
    • Machine learning and mahjong: an exploration of ai and human robot interaction (HRI)
    • Using facial mimicry to determine power and status differentials in group meetings over virtual platforms

    clean energy and image & acoustics signals analysis cohort 7 class photo 1

    clean energy and image & acoustics signals analysis cohort 7 class photo 2

    clean energy and image & acoustics signals analysis cohort 7 class photo 3

  • Cohort 6 (2019-2020)
    • A benchmark dataset for bluff detection in poker videos using facial analysis
    • Human motion synthesis to generate dance movements using neural networks
    • Randomized and realistic 3D avatar generation using generative adversarial networks
    • Robot navigation and object detection
    • Simulation of a robotic arm to assemble a tower of unique objects

    Image & Acoustics Signals Analysis cohort 6 class photo

  • Cohort 5 (2018-2019)
    • A Simplified Approach for Falsified Video Detection
    • Assigning Autonomous 2D Navigation Goals Utilizing Eye Gaze
    • Fatigue Detection Model Using Deep and Auxiliary Facial Feature Analysis
    • Find the Litter: a semi-supervised machine learning method for automatic litter detection
    • Multimodality Based Facial Expression Recognition

    Image & Acoustics Signals Analysis cohort 5 class photo

  • Cohort 4 (2017-2018)
    • Extracting and Applying Gaze Data for Gaze Pattern Identification
    • Holistic Identification of Scene Text with a General Image Classification CNN
    • Sorting Recyclable Waste to Prevent Contamination Using a Convolutional Neural Network

    Image & Acoustics Signals Analysis cohort 4 class photo

  • Cohort 3 (2016-2017)
    • 3D Object Detection for Visual Impairment
    • Using Artificial Occlusion to Facilitate Low-Resource Facial Recognition on Occluded Images
    • Gaze Patterns of Location Recognition/Non-Recognition
    • Facing Kinect Sensors to Differentiate Biological Gender Unobtrusively through Gait Detection

    Image & Acoustics Signals Analysis cohort 3 class photo

  • Cohort 2 (2015-2016)
    • Autism classification through gaze
    • Gesture recognition for automatic sign language interpretation
    • Pose estimation for automated control

    image & acoustics signals analysis cohort 2 class photo

Research Stream Collaborators

headshot of Kenneth Chiu

Kenneth Chiu

Associate Professor

School of Computing
EB Q15

Research Interests

  • High-performance computing
  • big data
  • bioinformatics
headshot of Scott A. Craver

Scott A. Craver

Associate Professor; Undergraduate Director

Electrical and Computer Engineering
ES 2310

Research Interests

  • Information security
  • Cryptology
  • Steganography
  • Watermarking and DRM systems
  • Security engineering