Featured

2023 Accepted/Invited Annual Meeting abstracts

  • AAPM Annual Meeting (Houston, TX • July 23 ‒ 27, 2023)
    1. Novel Functional Delta-Radiomics for Predicting Overall Survival in Lung Cancer Radiotherapy Using Cardiac FDG-PET Uptake
      Wookjin Choi, Yevgeniy Vinogradskiy
      Interactive ePoster Discussions: Sunday, July 23, 2023: 3:00 PM – 3:30 PM, GRBCC, Exhibit Hall | Forum 6
    2. Deep Learning Segmentation for Accurate GTV and OAR Segmentation in MR-Guided Adaptive Radiotherapy for Pancreatic Cancer Patients
      Wookjin Choi, Hamidreza Nourzadeh, Yingxuan Chen, Christopher G. Ainsley, Vimal K. Desai, Alexander A. Kubli, Yevgeniy Vinogradskiy, Maria Werner-Wasik, Adam Mueller, and Karen E. Mooney
    3. An MR-Conditional Motor for 1D Water Tank Measurements in an MR-Linac
      Karen E. Mooney, Sara A. Belko, Justin Williams, Andrew Gerry, Wookjin Choi, Maryellen Daley, and Robert S. Pugliese
    4. Quantitative Assessment of Dosimetric Effect of Using Alternative Spinalcord Delineation in Treatment Planning As Functions of Delineations, Setup Uncertainty and Planning Techniques Using an Alternative Truth Assessment Method
      Hashir Rashad, Abishek Karki, Jason Czak, Victor Gabriel Leandro Alves, Hamidreza Nourzadeh, Wookjin Choi, and Jeffrey V. Siebers
  • ASTRO Annual Meeting (San Diego, CA • October 1 ‒ 4, 2023)
    1. Novel Functional Radiomics for Prediction of Cardiac PET Avidity in Lung Cancer Radiotherapy
      Wookjin Choi, Yingqui Jia, Jennifer Kwak, Adam P. Dicker, Nicole L. Simone, Eugene Storozynsky, Varsha Jain, and Yevgeniy Vinogradskiy
      Quick Pitch Oral – QP 17b Phys 8: Outcome Prediction, October 3, 5:15PM-6:15PM
    2. Novel Deep Learning Segmentation Models for Accurate GTV and OAR Segmentation in MR-guided Adaptive Radiotherapy for Pancreatic Cancer Patients
      Wookjin Choi, Hamidreza Nourzadeh, Yingxuan Chen, Christopher G. Ainsley, Vimal K. Desai, Alexander A. Kubli, Yevgeniy Vinogradskiy, Karen E. Mooney, Maria Werner-Wasik, and Adam Mueller
      Poster Viewing Q&A Session Time: Tuesday, October 3, 4:00 PM-5:00 PM
  • INFORMS Annual Meeting (Phoenix, AZ • October 15 ‒ 18, 2023)
    1. Artificial Intelligence To Reduce Radiation-induced Cardiotoxicity In Lung Cancer Radiotherapy: A Novel Functional Radiomics Using Cardiac FDG-PET/CT
      Wookjin Choi, Adam Dicker, Yevgeniy Vinogradskiy

Longitudinal CBCT radiomics in Lung Cancer supported by Varian Medical Systems Inc.

Varian will support my research project entitled “Longitudinal CBCT radiomics analysis for lung cancer radiotherapy response and prognosis prediction” with $230,000 over 2 years. This is the first research grant from Varian to the Department of Radiation Oncology, Sidney Kimmel Medical College at Thomas Jefferson University. This project can potentially impact the clinical practice of lung cancer patients by using standard imaging modalities (CBCT and 4D-CBCT) to provide early prediction of prognosis and toxicity.

We will develop Jefferson Whole Lung CBCT radiomics framework that provides comprehensive lung analysis for early prediction of local control and normal tissue toxicity during lung radiation therapy. This project will provide predictive models for treatment response and prognosis using structural, morphologic, and functional radiomic features acquired during treatment with Varian CBCT imaging, as well as a unique opportunity to combine Varian technology (4D CBCT) with novel image processing techniques to acquire functional images. This project will also include a clinical study of the novel 4D functional CBCT imaging technique.

I am looking for a highly motivated postdoc to lead this project. If you are interested in working with us, please email me at “Wookjin.Choi@jefferson.edu.”

Clinically-Interpretable Radiomics

MICCAI’22 Paper | CMPB’21 Paper | CIRDataset

This library serves as a one-stop solution for analyzing datasets using clinically-interpretable radiomics (CIR) in cancer imaging (https://github.com/choilab-jefferson/CIR). The primary motivation for this comes from our collaborators in radiology and radiation oncology inquiring about the importance of clinically-reported features in state-of-the-art deep learning malignancy/recurrence/treatment response prediction algorithms. Previous methods have performed such prediction tasks but without robust attribution to any clinically reported/actionable features (see extensive literature on the sensitivity of attribution methods to hyperparameters). This motivated us to curate datasets by annotating clinically-reported features at the voxel/vertex level on public datasets (using our published advanced mathematical algorithms) and relating these to prediction tasks (bypassing the “flaky” attribution schemes). With the release of these comprehensively-annotated datasets, we hope that previous malignancy prediction methods can also validate their explanations and provide clinically-actionable insights. We also provide strong end-to-end baselines for extracting these hard-to-compute clinically-reported features and using these in different prediction tasks.

CIRDataset: A large-scale Dataset for Clinically-Interpretable lung nodule Radiomics and malignancy prediction [MICCAI’22]

Wookjin Choi1, Navdeep Dahiya2, and Saad Nadeem3
1 Department of Radiation Oncology, Thomas Jefferson University Hospital
2 School of Electrical and Computer Engineering, Georgia Institute of Technology
3 Department of Medical Physics, Memorial Sloan Kettering Cancer Center

Spiculations/lobulations, and sharp/curved spikes on the surface of lung nodules, are good predictors of lung cancer malignancy and hence, are routinely assessed and reported by radiologists as part of the standardized Lung-RADS clinical scoring criteria. Given the 3D geometry of the nodule and 2D slice-by-slice assessment by radiologists, manual spiculation/lobulation annotation is a tedious task and thus no public datasets exist to date for probing the importance of these clinically-reported features in the SOTA malignancy prediction algorithms. As part of this paper, we release a large-scale Clinically-Interpretable Radiomics Dataset, CIRDataset, containing 956 radiologist QA/QC’ed spiculation/lobulation annotations on segmented lung nodules from two public datasets, LIDC-IDRI (N=883) and LUNGx (N=73). We also present an end-to-end deep learning model based on multi-class Voxel2Mesh extension to segment nodules (while preserving spikes), classify spikes (sharp/spiculation and curved/lobulation), and perform malignancy prediction. Previous methods have performed malignancy prediction for LIDC and LUNGx datasets but without robust attribution to any clinically reported/actionable features (due to known hyperparameter sensitivity issues with general attribution schemes). With the release of this comprehensively-annotated dataset and end-to-end deep learning baseline, we hope that malignancy prediction methods can validate their explanations, benchmark against our baseline, and provide clinically-actionable insights. Dataset, code and pre-trained models are available in this repository.

Dataset

The first CIR dataset, released here, contains almost 1000 radiologist QA/QC’ed spiculation/lobulation annotations (computed using our published LungCancerScreeningRadiomics library and QA/QC’ed by a radiologist) on segmented lung nodules for two public datasets, LIDC (with visual radiologist malignancy RM scores for the entire cohort and pathology-proven malignancy PM labels for a subset) and LUNGx (with pathology-proven size-matched benign/malignant nodules to remove the effect of size on malignancy prediction).

Clinically-interpretable spiculation/lobulation annotation dataset samples; the first column – input CT image; the second column – overlaid semi-automated/QA/QC’ed contours and superimposed area distortion maps (for quantifying/classifying spikes, computed from spherical parameterization — see our LungCancerScreeninigRadiomics Library); the third column – 3D mesh model with vertex classifications, red: spiculations, blue: lobulations, white: nodule base.

End-to-End Deep Learning Nodule Segmentation, Spikes’ Classification, and Malignancy Prediction Model

We also release our multi-class Voxel2Mesh extension to provide a strong benchmark for end-to-end deep learning lung nodule segmentation, spikes’ classification (lobulation/spiculation), and malignancy prediction; Voxel2Mesh is the only published method to our knowledge that preserves sharp spikes during segmentation and hence its use as our base model. With the release of this comprehensively-annotated dataset, we hope that previous malignancy prediction methods can also validate their explanations/attributions and provide clinically-actionable insights. Users can also generate spiculation/lobulation annotations from scratch for LIDC/LUNGx as well as new datasets using our LungCancerScreeningRadiomics library.

Depiction of end-to-end deep learning architecture based on multi-class Voxel2Mesh extension. The standard UNet based voxel encoder/decoder (top) extracts features from the input CT volumes while the mesh decoder deforms an initial spherical mesh into increasing finer resolution meshes matching the target shape. The mesh deformation utilizes feature vectors sampled from the voxel decoder through the Learned Neighborhood (LN) Sampling technique and also performs adaptive unpooling with increased vertex counts in high curvature areas. We extend the architecture by introducing extra mesh decoder layers for spiculation and lobulation classification. We also sample vertices (shape features) from the final mesh unpooling layer as input to Fully Connected malignancy prediction network. We optionally add deep voxel-features from the last voxel encoder layer to the malignancy prediction network

Results

The following tables show the expected results of running the pre-trained ‘Mesh Only’ and ‘Mesh+Encoder’ models.

Table1. Nodule (Class0), spiculation (Class1), and lobulation (Class2) peak classification metrics

Training
Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑
Class0 Class1 Class2 Class0 Class1 Class2
Mesh Only 0.009 0.010 0.013 0.507 0.493 0.430
Mesh+Encoder 0.008 0.009 0.011 0.488 0.456 0.410
Validation
Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑
Class0 Class1 Class2 Class0 Class1 Class2
Mesh Only 0.010 0.011 0.014 0.526 0.502 0.451
Mesh+Encoder 0.014 0.015 0.018 0.488 0.472 0.433
Testing LIDC-PM N=72
Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑
Class0 Class1 Class2 Class0 Class1 Class2
Mesh Only 0.011 0.011 0.014 0.561 0.553 0.510
Mesh+Encoder 0.009 0.010 0.012 0.558 0.541 0.507
Testing LUNGx N=73
Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑
Class0 Class1 Class2 Class0 Class1 Class2
Mesh Only 0.029 0.028 0.030 0.502 0.537 0.545
Mesh+Encoder 0.017 0.017 0.019 0.506 0.523 0.525

Table 2. Malignancy prediction metrics.

Training
Network AUC Accuracy Sensitivity Specificity F1
Mesh Only 0.885 80.25 54.84 93.04 65.03
Mesh+Encoder 0.899 80.71 55.76 93.27 65.94
Validation
Network AUC Accuracy Sensitivity Specificity F1
Mesh Only 0.881 80.37 53.06 92.11 61.90
Mesh+Encoder 0.808 75.46 42.86 89.47 51.22
Testing LIDC-PM N=72
Network AUC Accuracy Sensitivity Specificity F1
Mesh Only 0.790 70.83 56.10 90.32 68.66
Mesh+Encoder 0.813 79.17 70.73 90.32 79.45
Testing LUNGx N=73
Network AUC Accuracy Sensitivity Specificity F1
Mesh Only 0.733 68.49 80.56 56.76 71.60
Mesh+Encoder 0.743 65.75 86.11 45.95 71.26

Lung Cancer Screening Radiomics

A comprehensive framework for lung cancer screening radiomics using LIDC-IDRI and LUNGx dataset.

  • Data preprocessing – download data, conversion, etc.
  • Radiomics feature extraction including spiculation features
  • AutoML model building and validation

Source code https://github.com/choilab-jefferson/LungCancerScreeningRadiomics

Publications

  1. Wookjin Choi, Jung Hun Oh, Sadegh Riyahi, Chia-Ju Liu, Feng Jiang, Wengen Chen, Charles White, Andreas Rimner, James G. Mechalakos, Joseph O. Deasy, and Wei Lu, “Radiomics analysis of pulmonary nodules in low-dose CT for early detection of lung cancer”, Medical Physics, Vol. 45, No. 4, pp. 1537-1549, April 2018. https://doi.org/10.1002/mp.12820
  2. Wookjin Choi, Saad Nadeem, Sadegh Riyahi, Joseph O. Deasy, Allen Tannenbaum, Wei Lu, “Reproducible and Interpretable Spiculation Quantification for Lung Cancer Screening.” Computer methods and programs in biomedicine. 200 2021. https://doi.org/10.1016/j.cmpb.2020.105839

Hiring a Postdoctoral Fellow

Postdoctoral Fellow – Developing Clinically Interpretable Medical Imaging AI in Radiation Therapy

https://recruit.jefferson.edu/psp/hcmp/EMPLOYEE/HRMS/c/HRS_HRAM_FL.HRS_CG_SEARCH_FL.GBL?Page=HRS_APP_JBPST_FL&Action=U&FOCUS=Applicant&SiteId=1&JobOpeningId=9272548&PostingSeq=1
  • PI: Wookjin Choi, Ph.D. <Wookjin.Choi@jefferson.edu>
    Assistant Professor of Radiation Oncology, Thomas Jefferson University
  • 2 Years

Responsibilities

POST-DOCTORAL POSITION, DEPARTMENT OF RADIATION ONCOLOGY: Thomas Jefferson University is now accepting applications for a post-doctoral fellow in the Department of Radiation Oncology with the Choi lab.  The post-doctoral position is for developing AI techniques for image-guided radiation therapy and clinical outcome prediction and decision-making using radiomics, deep learning, and other computationally intensive techniques. Trainees must have the opportunity to carry out supervised biomedical research with the primary objective of developing or extending their research skills and knowledge in preparation for an independent research career.

This is an exciting opportunity to work on an emerging research project funded by Sidney Kimmel Cancer Center on the incorporation of medical imaging AI into radiation therapy and clinical care for cancer patients. The research focuses on the development of image analysis tools for cancer imaging, as well as clinically interpretable radiomics features and deep learning models for clinical outcome prediction and decision making. The post-doctoral fellow will have an opportunity to collaborate with Thomas Jefferson faculty, national and international collaborators, and work alongside investigators at the NCI Quantitative Imaging Network.

The entire Thomas Jefferson Medical Physics Division currently consists of 22 physicists and eight physics residents. The Jefferson physics faculty lead a highly-impactful and diverse research program. The group currently has ongoing funded projects by the National Cancer Institute, vendor-funded research, as well as Sidney Kimmel Cancer Center funding. System-wide equipment includes a ViewRay MRI-Linac, Varian TrueBeams, and Elekta Agility Linacs. The department has advanced scripting capabilities with multiple Radiation Oncology software packages including Mosaiq, Eclipse, and MIM Software.

Thomas Jefferson University is an Equal Opportunity Employer.  Jefferson values diversity and encourages applications from women, members of minority groups, LGBTQ individuals, disabled individuals, and veterans. Applicants should forward a curriculum vitae and a statement of interest to the administrative assistant, Juli Johnson at julianne.johnson@jefferson.edu.

Qualifications

Candidates must have a Ph.D. in Computer Science, Medical Physics, Physics, Mathematics, Electrical Engineering, Biomedical Engineering, or related field required. The ideal candidate will be highly interested in an academic Medical Physics career, have strong computational skills, and seek out a highly collaborative environment. Based on the interest of the post-doctoral fellow; opportunities will be provided to obtain clinical experience, treatment planning experience, as well as mentorship on clinical trial design and statistical modeling methods.

Conditions of Employment

Covid Vaccination is a requirement for employment at Jefferson for employees working at Jefferson’s clinical entities or at the University.  If you are not currently vac

PathCNN: interpretable convolutional neural networks for survival prediction and pathway analysis applied to glioblastoma

Jung Hun Oh, Wookjin Choi, Euiseong Ko, Mingon Kang, Allen Tannenbaum, Joseph O Deasy

The authors wish it to be known that, in their opinion, Jung Hun Oh and Wookjin Choi should be regarded as Joint First Authors.

https://academic.oup.com/bioinformatics/article/37/Supplement_1/i443/6319702

An illustration of biological interpretation. (A) Grad-CAM procedure to generate class activation maps. The two images on the left bottom represent an example of the class activation maps for a sample in the cohort, which were generated from Grad-CAM procedure; (B) statistical analysis to identify significantly different pathways between the LTS and non-LTS groups. LTS, long-term survival; CNN, convolutional neural network; ReLU, rectified linear unit

Abstract

Motivation

Convolutional neural networks (CNNs) have achieved great success in the areas of image processing and computer vision, handling grid-structured inputs and efficiently capturing local dependencies through multiple levels of abstraction. However, a lack of interpretability remains a key barrier to the adoption of deep neural networks, particularly in predictive modeling of disease outcomes. Moreover, because biological array data are generally represented in a non-grid structured format, CNNs cannot be applied directly.

Results

To address these issues, we propose a novel method, called PathCNN, that constructs an interpretable CNN model on integrated multi-omics data using a newly defined pathway image. PathCNN showed promising predictive performance in differentiating between long-term survival (LTS) and non-LTS when applied to glioblastoma multiforme (GBM). The adoption of a visualization tool coupled with statistical analysis enabled the identification of plausible pathways associated with survival in GBM. In summary, PathCNN demonstrates that CNNs can be effectively applied to multi-omics data in an interpretable manner, resulting in promising predictive power while identifying key biological correlates of disease.Availability and implementation

The source code is freely available at: https://github.com/mskspi/PathCNN.

Fourth place Winner on AI Tracks at Sea Challenge

We won 4th place in the Artificial Intelligence (AI) Tracks at Sea Challenge. https://www.challenge.gov/challenge/AI-tracks-at-sea/
This national competition is organized by the U.S. Navy.

VSU TrojanOne Team: Jose Diaz, Curtrell Trott, Advisor: Ju Wang, Wookjin Choi

The $200,000 prize was distributed among five winning teams, which submitted full working solutions, and three runners-up, which submitted partial working solutions. The monetary prize will be awarded to the school the corresponding team attends:

Challenge Winners

Teams participating in the AI Tracks at Sea Challenge spanned collegiate institutions from east to west U.S. coasts, from both public and private colleges and universities. Collectively, the student submissions for the challenge represent various types of STEM research institutions, Ivy League Schools, Historically Black Colleges and Universities (HBCU) and Hispanic Serving Institutes (HSI). Of the challenge teams, 26% were comprised of students from HBCUs and 16% of the teams attend HSIs.

“With 94% of the competitors attending colleges and universities outside of California, this challenge served as an avenue to make broader impacts in STEM,” said Yolanda Tanner, Naval Information Warfare Systems Command (NAVWAR) STEM Federal Action Officer and NIWC Pacific Internship and Fellowship project manager. “It was also a means by which students could further develop their STEM skills while working collaboratively to solve a real-world naval problem.”

Florida, North Carolina, and Texas had the largest population of participating collegiate teams.

VSU Team Takes Fourth Place in AI Tracks at Sea Naval Challenge

Automatic motion tracking system for analysis of insect behavior

Darrin Gladman, Jehu Osegbe, Wookjin Choi*, and Joon Suk Lee “Automatic motion tracking system for analysis of insect behavior”, Proc. SPIE 11510, Applications of Digital Image Processing XLIII, 115102W (21 August 2020); https://doi.org/10.1117/12.2568804

*Corresponding author

Abstract

We present a multi-object tracking system to track small insects such as ants and bees. Motion-based object tracking recognizes the movements of objects in videos using information extracted from the given video frames. We applied several computer vision techniques, such as blob detection and appearance matching, to track ants. Moreover, we discussed different object detection methodologies and investigated the various challenges of object detection, such as illumination variations and blob merge/split. The proposed system effectively tracked multiple objects in various environments.

Reproducible and Interpretable Spiculation Quantification for Lung Cancer Screening

Choi, W., Nadeem, S., Alam, S. R., Deasy, J. O., Tannenbaum, A., & Lu, W. (2020). Reproducible and Interpretable Spiculation Quantification for Lung Cancer Screening. Computer Methods and Programs in Biomedicine, 105839. https://doi.org/10.1016/j.cmpb.2020.105839

Source codes: https://github.com/choilab-jefferson/LungCancerScreeningRadiomics


Highlights

  • A novel interpretable spiculation feature is presented, computed using the area distortion metric from spherical conformal (angle-preserving) parameterization.
  • A simple one-step feature and prediction model is introduced which only uses our interpretable features (size, spiculation, lobulation, vessel/wall attachment) and has the added advantage of using weak-labeled training data.
  • A semi-automatic segmentation algorithm is also introduced for more accurate and reproducible lung nodule as well as vessel/wall attachment segmentation. This leads to more accurate spiculation quantification because the attachments can be excluded from spikes on the lung nodule surface (triangular mesh) data.
  • Using just our interpretable features (size, attachment, spiculation, lobulation), we were able to achieve AUC=0.82 on public Lung LIDC dataset and AUC=0.76 on public LUNGx dataset (the previous LUNGx best being AUC=0.68).
  • State-of-the-art correlation is achieved between our spiculation score (the number of spiculations, Ns) and radiologists spiculation score (ρ = 0.44).

Abstract

Spiculations are important predictors of lung cancer malignancy, which are spikes on the surface of the pulmonary nodules. In this study, we proposed an interpretable and parameter-free technique to quantify the spiculation using area distortion metric obtained by the conformal (angle-preserving) spherical parameterization. We exploit the insight that for an angle-preserved spherical mapping of a given nodule, the corresponding negative area distortion precisely characterizes the spiculations on that nodule. We introduced novel spiculation scores based on the area distortion metric and spiculation measures. We also semi-automatically segment lung nodule (for reproducibility) as well as vessel and wall attachment to differentiate the real spiculations from lobulation and attachment. A simple pathological malignancy prediction model is also introduced. We used the publicly-available LIDC-IDRI dataset pathologists (strong-label) and radiologists (weak-label) ratings to train and test radiomics models containing this feature, and then externally validate the models. We achieved AUC = 0.80 and 0.76, respectively, with the models trained on the 811 weakly-labeled LIDC datasets and tested on the 72 strongly-labeled LIDC and 73 LUNGx datasets; the previous best model for LUNGx had AUC = 0.68. The number-of-spiculations feature was found to be highly correlated (Spearman’s rank correlation coefficient ) with the radiologists’ spiculation score. We developed a reproducible and interpretable, parameter-free technique for quantifying spiculations on nodules. The spiculation quantification measures was then applied to the radiomics framework for pathological malignancy prediction with reproducible semi-automatic segmentation of nodule. Using our interpretable features (size, attachment, spiculation, lobulation), we were able to achieve higher performance than previous models. In the future, we will exhaustively test our model for lung cancer screening in the clinic.