Jun Hao Liew

I am a senior research scientist at TikTok/ByteDance Research. My research focus is controllable diffusion-based generative models. Before joining TikTok, I worked as a research fellow at NUS. Prior to that, I did my Ph.D. with A/P Sim-Heng Ong, Dr. Wei Xiong and Dr. Jiashi Feng in NUS.

I am actively looking for research interns and collaborators. Please feel free to drop me an email if you are interested.

Email  /  Google Scholar  /  LinkedIn  /  Twitter  /  Github

Publications

We introduce GigaTok, the first method for scaling visual tokenizers to 3 billion parameters.

We systematically compare SAIL’s properties—including scalability, cross-modal information flow patterns, and visual representation capabilities—with those of modular MLLMs.

We train a fast (<1s) and accurate drag-based image editing model by learning from video supervision.

This work presents Diffusion GLA, the first exploration for diffusion backbone with linear attention transformer.

Given an input mesh, MagicArticulate first generates skeleton autoregressively and then predicts skinning weights, making it articulation-ready.

We present ClassDiffusion to mitigate the weakening of compositional ability during personalization tuning.

We propose AvatarStudio, a coarse-to-fine generative model that generates explicit textured 3D meshes for animatable human avatars.

We improve the appearance quality of MagicAnimate by introducing regional supervision and explicit modeling of motion blur.

We present Creativity-VLM, a vision-language assistant that can translate coarse editing hints (e.g., "spring") into precise, actionable instructions for image editing.

We present PeRFlow, a flow-based method for accelerating diffusion models.

MagicVideo-V2 integrates text-to-image model, video motion generator, reference image embedding module and frame interpolation module into an end-to-end video generation pipeline.

We present Symplectic Adjoint Guidance (SAG) to obtain accurate gradient guidance for training-free guided sampling in diffusion models.

We present DragDiffusion, which extends interactive point-based image editing to large-scale pretrained diffusion models.

We propose MagicAnimate, a diffusion-based human image animation framework that aims at enhancing temporal consistency, preserving reference image faithfully, and improving animation fidelity.

XAGen is a 3D-aware generative model that enables human synthesis with high-fidelity appearance and geometry, together with disentangled controllability for body, face, and hand.

We present MixVal, a model selection method that operates solely with unlabeled target data during inference to select the best UDA model for the target domain.

We present SegRefiner, a universal segmentation refinement model that is applicable across diverse segmentation models and tasks (e.g., semantic, instance, and dichotomous segmentation).

MagicEdit explicitly disentangles the learning of appearance and motion to achieve high-fidelity and temporally coherent video editing.

MagicAvatar is a multi-modal framework that is capable of converting various input modalities — text, video, and audio — into motion signals that subsequently generate/ animate an avatar.

MagicProp employs the edited frame as an appearance reference and generates the remaining frames using an autoregressive rendering approach.

We developed a fast open-vocabulary semantic segmentation model that can perform comparably or better without the extra computational burden of the CLIP image encoder during inference.

We conduct an empirical study on the scaling capability of masked image modeling (MIM) methods for visual recognition.

Associating spatially-consistent grouping of self-supervised vision models with text-supervised semantic segmentation.

We propose a 3D-aware portrait video GAN, PV3D, which is capable to generate a large variety of 3D-aware portrait videos with high-quality appearance, motions, and 3D geometry.

We explored a new task called semantic mixing, aiming at mixing two different semantics to create a new concept (e.g., tiger and rabbit).

Our Slim Scissors enables quick extraction of elongated thin parts by simply brushing some coarse scribbles.

We develop a novel learning-based aggregation method that improves upon SOLO by leveraging the rich neighboring information while maintaining the architectural efficiency.

We identify the issue of indirect information propagation between deeper and shallower layers in FPN-based saliency methods and present a cross-layer communication mechanism for better salient object detection.

We present the first single-stage model for multi-person body mesh recovery.

We revisit the use of superpixels for active learning in segmentation and demonstrate that the inappropriate choice of cost measure may cause the effectiveness of the superpixel-based approach to be under-estimated.

With our proposed attentive deformation mechanism and segment-wise matching scheme, our contour-based instance segmentation model DANCE performs comparably to existing top-performing pixel-based models.

We present ThinObject-5K, a large-scale dataset for segmentation of thin elongated objects. We also designed a three-stream network that integrates high-resolution boundary information with fixed resolution semantic contexts for effective segmentation of thin parts.

We propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach.

We present a simple Inside-Outside Guidance (IOG) that takes 3 clicks for efficient interactive segmentation.

We propose a deep yet light-weight saliency inference module that adopts a multi-dilated depth-wise convolution architecture for salient object detection.

MultiSeg generates a set of scale-varying proposals that conform to the user input for interactive segmentation.

PANet introduces a prototype alignment regularization between support and query for better generalization on few-shot segmentation.

We present FSENet to tackle the class imbalance and inter-class interference problem in multi-label brain tumor segmentation.

RIS-Net expands the field-of-view of the given input clicks to capture the local regional information surrounding them for local refinement.