Vision and System Design Lab

Who We Are

Welcome to HKUST Vision and System Design Lab. The focus of our lab includes design, optimization and compression of artificial intelligence (AI) models, as well as architecture and design of AI chips/systems for energy-efficient training and inference of such AI models. Currently, we place strong focus on multimodal large foundation models for computer vision, vision-language, and medical applications. Our Lab also collaborates closely with InnoHK AI Chip Center for Smart Emerging Systems on co-design and co-optimization of AI systems across application, algorithm and hardware layers.

News

  • May 2025.  Exploiting the Memory-Compute-Coupling Feature for CIM Accelerator Design Optimization is accepted by TCAD 2025. This article proposes MCC-DSE, a DSE framework that tackles the memory-compute-coupling challenges in CIM accelerators by identifying the optimal memory-compute ratio and employing a three-axis mapping to enable fine-grained optimization. The results demonstrate enhanced performance and efficiency across different NN models and area budgets, providing insights into future CIM accelerator design.
  • March 2025.  Rethinking Boundary Detection in Deep Learning-Based Medical Image Segmentation is accepted by Medical Image Analysis. In this paper, a novel network architecture, referred to as Convolution, Transformer, and Operator (CTO), is proposed. CTO employs a combination of Convolutional Neural Networks, Vision Transformer, and an explicit boundary detection operator to achieve high recognition accuracy while maintaining an optimal balance between accuracy and efficiency.
  • February 2025.  SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training is accepted by CVPR 2025. This paper builds an extremely small and fast T2I model that generates high-resolution and high-quality images on mobile platforms. Our model, for the first time, demonstrates the generation of 1024x1024 px images on a mobile device in 1.2-2.3 seconds.
  • February 2025.  MedIAnomaly: A Comparative Study of Anomaly Detection in Medical Images is accepted by Medical Image Analysis. This paper builds a benchmark with unified comparison. Seven medical datasets with five image modalities, including chest X-rays, brain MRIs, retinal fundus images, dermatoscopic images, and histopathology whole slide images, are curated for extensive evaluation.
  • January 2025.  Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection is accepted by ICLR 2025. In this work, we propose CCKT-Det trained without any extra supervision. The proposed framework constructs a cyclic and dynamic knowledge transfer from language queries and visual region features extracted from VLMs, which forces the detector to closely align with the visual-semantic space of VLMs.
  • January 2025.  Memory Efficient Transformer Adapter for Dense Predictions is accepted by ICLR 2025. In this work, we propose META, a simple and fast ViT adapter that can improve the model's memory efficiency and decrease memory time consumption by reducing the inefficient memory access operations.
  • January 2025.  Boosting Convolution with Efficient MLP-Permutation for Volumetric Medical Image Segmentation is accepted by TMI. In this paper, we propose a novel permutable hybrid network for Vol-MedSeg, named PHNet, which capitalizes on the strengths of both convolution neural networks and MLP.
  • January 2025.  Dynamic Subcluster-Aware Network for Few-Shot Skin Disease Classification is publicized on TNNLS. In this paper, we focus on learning a high-quality feature encoder that captures the unique subclustered representations within each disease class, enabling better characterization of feature distributions.
  • December 2024.  Generalized Task-Driven Medical Image Quality Enhancement with Gradient Promotion is accepted by TPAMI. In this paper, we propose a generalized gradient promotion training strategy for task-driven image quality enhancement of medical images.
  • October 2024.  A 28nm 0.22μJ/Token Memory-Compute-Intensity-Aware CNN-Transformer Accelerator with Hybrid-Attention-Based Layer-Fusion and Cascaded Pruning for Semantic-Segmentation is accepted by ISSCC 2025. In this paper, we propose a hybrid attention mechanism coupled with a KV-weight-reused scheduler to fuse the layers between convolution and attention. Additionally, we introduce a cascaded feature map pruning strategy to achieve unified convolution-attention pruning.
  • September 2024.  Prof. Tim Cheng has been awarded the "CCF海外科技人物奖" in recognition of his outstanding achievements in the field of computer science and his contributions to the advancement of computer science in China.
  • September 2024.  RoLoRA: Finetuning Outlier-free Model with Rotation for Weight-Activation Quantization is accepted by EMNLP 2024. This paper presents RoLoRA, an efficient LLM fine-tuning method preserving the outlier-free characteristics brought by rotation operations for effective weight-activation quantization.
  • September 2024.  Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning is accepted by EMNLP 2024. In this paper, we introduce a coarse-to-fine pruner to maximize the input of effective CoT examples, thereby improving LLM mathematical reasoning capability.
  • September 2024.  Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision is accepted by TMLR. In this paper, we provide an analysis of the underlying difficulty of transformer quantization in the view of variation. A multi-crop knowledge distillation-based quantization method is proposed.
  • August 2024.  Efficient Quantization-aware Training with Adaptive Coreset Selection is accepted by TMLR. This work introduces Quantization-Aware Adaptive Coreset Selection (ACS), a method that uses error vector and disagreement scores to select crucial training samples, significantly improving the training efficiency.
  • August 2024.  Labeled-to-Unlabeled Distribution Alignment for Partially-Supervised Multi-Organ Medical Image Segmentation is accepted by Medical Image Analysis. In this paper, to address the problem of distribution mismatch, we propose a labeled-to-unlabeled distribution alignment framework that aligns feature distributions and enhances discriminative capability.
  • August 2024.  Vessel-Promoted OCT to OCTA Image Translation by Heuristic Contextual Constraints is accepted by Medical Image Analysis. We propose a novel framework, TransPro, that translates 3D OCT images into exclusive 3D OCTA images using an image translation pattern. Our main objective is to address two issues in existing image translation baselines, namely, the aimlessness in the translation process and incompleteness of the translated object.
  • June 2024.  Iterative Online Image Synthesis via Diffusion Model for Imbalanced Classification is accepted by MICCAI 2024. We introduce an Iterative Online Image Synthesis (IOIS) framework to address the class imbalance problem in medical image classification.
  • June 2024.  Rethinking Autoencoders for Medical Anomaly Detection from A Theoretical Perspective is accepted by MICCAI 2024. We for the first time provide a theoretical foundation for AE-based reconstruction methods in anomaly detection. By leveraging information theory, we elucidate the principles of these methods and reveal that the key to improving their performance lies in minimizing the information entropy of latent vectors.
  • May 2024.  Aligning Medical Images with General Knowledge from Large Language Models is early accepted by MICCAI 2024. We proposed ViP, a novel visual symptom-guided prompt learning framework for medical image analysis, which facilitates general knowledge transfer from CLIP. ViP mainly consists of two key components: a visual symptom generator and a dual-prompt network.
  • May 2024.  LENAS: Learning-based Neural Architecture Search and Ensemble for 3D Radiotherapy Dose Prediction is accepted by IEEE Transactions On Cybernetics. We proposed a novel learning-based ensemble approach named LENAS, which integrates neural architecture search with knowledge distillation for 3D radiotherapy dose prediction.
  • April 2024.  DoRA: Weight-Decomposed Low-Rank Adaptation is accepted by ICML 2024. We first introduced a novel weight decomposition analysis to investigate the inherent differences between FT and LoRA. Aiming to resemble the learning capacity of FT from the findings, we propose Weight-Decomposed LowRank Adaptation (DoRA).
  • March 2024.  BoNuS: Boundary Mining for Nuclei Segmentation with Partial Point Labels appears at IEEE Transactions on Medical Imaging. We proposed a novel boundary mining framework for nuclei segmentation, named BoNuS, which simultaneously learns nuclei interior and boundary information from the point labels.
  • March 2024.  Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers is accepted by DAC 2024. This paper proposed a genetic LUT-Approximation algorithm namely GQA-LUT that can automatically determine the parameters with quantization awareness. GQA-LUT enables the employment of INT8-based LUT-Approximation that achieves an area savings of 81.3~81.7% and a power reduction of 79.3~80.2% compared to the high-precision FP/INT 32 alternatives.
  • December 2023.  AdaP-CIM: Compute-in-Memory Based Neural Network Accelerator using Adaptive Posit is accepted by DATE 2024.
  • November 2023.  CAE-GReaT: Convolutional-Auxiliary Efficient Graph Reasoning Transformer for Dense Image Predictions is accepted by International Journal of Computer Vision. We proposed an auxiliary and integrated network architecture, named Convolutional-Auxiliary Efficient Graph Reasoning Transformer, which joints strengths of both CNNs and ViT into a uniform framework.
  • November 2023.  Dynamic Sub-Cluster-Aware Network for Few-Shot Skin Disease Classification is accepted by IEEE Transactions on Neural Networks and Learning Systems. This paper addresses the problem of few-shot skin disease classification by introducing a novel approach called the Sub-Cluster-Aware Network (SCAN) that enhances accuracy in diagnosing rare skin diseases.
  • October 2023.  Nuclei segmentation with point annotations from pathology images via self-supervised learning and co-training is accepted by Medical Image Analysis. We proposed a weakly-supervised learning method for nuclei segmentation that only requires point annotations for training.
  • July 2023.  Compete to Win: Enhancing Pseudo Labels for Barely-supervised Medical Image Segmentation is accepted by IEEE Transactions on Neural Networks and Learning Systems. We proposed a novel Compete-to-Win method to enhance the pseudo label quality. In contrast to directly using one model's predictions as pseudo labels, our key idea is that high-quality pseudo labels should be generated by comparing multiple confidence maps produced by different networks to select the most confident one.
    • Read More

    Highlights

    Our publications

    Our Publications

    Our lab is at the forefront of cutting-edge AI research, covering a wide range of promising topics including AI chips/systems, compute-in-memory, electronic design automation, computer vision, vision-language, tiny machine learning, co-design, large foundational models, and medical image analysis.

    See our publications
    Our Research

    Our Research

    Our research aims to make advanced AI technologies more effective and efficient, enabling everyone to enjoy the convenience and pleasure brought by these transformative technologies that will shape the future.

    Browse our research
    Our Demos

    Our Demos

    Technology should never be limited to papers and slides. We strongly believe that our success in projects and demos is valuable and can drive potential solutions to practical problems.

    Watch our demos
    Our Team

    Our Team

    We are a team of passionate researchers who are dedicated to pushing the forefront of artificial intelligence accelerator. We are committed to creating an inclusive environment for research and acknowledge the importance of diverse knowledge backgrounds in the discovery process.

    Meet our team