During this senior project season, I'll be working to assist with research tasks at the OHSU department of interventional radiology. Specifically, I've been directed to work on Dr. Brian Park's research into possible applications of augmented reality technology in tumor ablation procedures.
A picture of OHSU, but I'm not actually there because I'm working virtually.
My tasks currently revolve around developing shape recognition systems in python for superimposing virtual models onto actual patients by recognizing vision tags in DICOM image files. You're probably thinking to yourself, wow, what a cool story bro, that's a lot of buzzwords that mean absolutely nothing to me. In less-pretentious terms, I'm just finding a way to put a 3-D model of the inside of someone's body, organs, and all, onto someone's actual body to make sure we know where all the important squishy bits are during surgery.
The idea is to make surgery less like "playing pin the tail on the donkey", and more like "playing pin the tail on the donkey without a blindfold on".
Writing this project completely from scratch would be night impossible, but with the help of several python packages (especially OpenCV-python and PyDicom) a Catlin Comp Sci student like me can actually write useful code. This is in no small part thanks to the always amazing Andrew Merril and the fantastically designed Catlin Computer Science curriculum.
There are two-element to successfully identifying vision tags in DICOM files, which are produced from X-Rays, MRIs, and other medical imaging tools. First, I have to convert the files to a generally readable format, for example, an array of pixel data, and then write out the actual image recognition code. The aforementioned PyDicom and opencv-python packages are used for the first and second legs of the process respectively.
Talking about code is pretty boring, so I've just put a screenshot of some of my code above for you to look at. Feel free to ask me about the technical details of what I'm doing in the comment section below.
I've always thought that biology was a pretty neat area of study, but unfortunately, I was never particularly good at studying the subject. Luckily, I figured that if I could just apply some of my technical skills to some sort of medical research initiative, I could have the best of both worlds. The process that I went through to actually pull the project together involved sending a large number of emails to various entities at OHSU. I'm incredibly fortunate to have two fantastic mentors in Dr. Park, my project supervisor, and the esteemed Marion Edens, my catlin mentor.
3 goals:
- make a meaningful contribution to Dr. Park's research
- learn some more about applications of AR in the medical field
- establish whether a career in research is something I’d want to pursue
Yes, Eric! Big brain time
ReplyDeleteYour project sounds really interesting, even if I have no clue what you mean when talking about coding. I like your metaphor about pinning the tail on the donkey. This project sounds as if it will really help surgeons locate the bits they need to find.
ReplyDeleteThank you, Eric. Your description of what you are doing as "pin the tail on the donkey without a blindfold" is awesome. Are you creating a virtual 3D model? Is this something where you could create an augmented reality for the surgeon?
ReplyDeleteHello Becky,
DeleteThanks for the comment! I'm actually just writing a python program to utilize shape recognition software in order to identify certain sterile tags in medical imaging scans, allowing these scans to be accurately mapped onto a patients body to reveal the precise locations of tumors, organs, etc. etc. I'm not actually creating the 3D models themselves since those can be rendered automatically without human input. Hopefully that answers your question.
Nice work Eric, very cool!
ReplyDeleteEric, this is absolutely fascinating work. I look forward to reading more of your blog!
ReplyDelete