27" Retina iMac First Impressions

I just received my 27" Retina iMac yesterday and figured I would post my impression so far.

The Display

The most exciting thing about the new 27" Retina iMac is the display.  It runs as 5120x2880 resolution which is 4x the number of pixels as the Apple Thunderbolt Display and non retina 27" iMac.  The higher resolution of the retina display is of specific interest to diagnostic radiology as it will allow high resolution images such as chest x-rays and mammograms to be displayed at full resolution.  Displaying these images at full resolution is important because it eliminates the need for the radiologist to zoom and pan the image thus saving time. 

After starting up the Retina iMac for the first time, I was a bit disappointed because I didn't really notice that much of a difference compared to the Thunderbolt Display. It certainly wasn't any worse - the display is just as beautiful and bright as any other Apple display, but nothing stood out as amazingly got to have it better. It did "feel" a bit different but it was hard to quantify.  I knew it was higher resolution so looked around for more detailed icons and text and convinced myself it was better when I looked - but the difference is quite subtle.  

One thing I was relieved to discover is that the display does in fact run at 60hz.  I had tried out another display earlier this year that could only do 30hz and it was unusable for me.  I had read one review about the retina display claiming that it only did 30hz but that is definitely not the case.

Interestingly enough, I have really noticed the improvement in resolution while writing this blog entry.  The text is much sharper and it feels like reading a book.  I have the feeling that the benefits of the display will be clearer when after I get used to it and then look at a non retina display.  I had a similar experience with the retina iPhone - not initially impressed but very hard to go back to a non retina device.

The Video Card

The Retina iMac comes with an AMD Radeon R9 M290X with 2 GB RAM or an AMD Radeon R9 M295X with 4 GB RAM.  There was some concern online about how well the system would handle 4x the pixels to push around.  I am happy to report that it handles it just fine - I haven't noticed any drop in performance - in fact, it feels faster than my 15" Retina Macbook Pro did connected to the thunderbolt display.  I have noticed two issues so far - one is that youtube videos would not playback at full screen (I could hear audio, but the screen was black).  This one seemed to have fixed itself, so it may have just been a glitch.  The other is that World of Warcraft seems to stutter in some places. This stuttering is really annoying and feels like a software bug in the game or video driver that will hopefully be fixed in the near future.

The CPU

According to Geekbench, the Retina iMac is the fastest Mac available today for single core work and is only beat by the high end Mac Pro for multicore work.  The machine certainly feels fast, I was playing WoW with two Windows 7 virtual machines running in the background and didn't even notice it.  The only drawback to this system is the fan is louder than I like.  The fan turned on while I was playing WoW and the noise was noticeable.  Not as loud as the fan on my MBP, but still loud enough to be annoying.

I also tried out the window/level operation on a full resolution mammogram using cornerstone and it was able to keep up at about 20 FPS which is a bit lower than I would like but fast enough for most users.

The Flash Drive

The new iMac comes with a PCIe Flash Drive.  I was hoping that it would have similar performance to the rMBP but was disappointed to learn that it is quite a bit slower.  It is faster than a SATA SSD drive, but not by much.  Here is a screenshot from running blackmagic:

Segmenting the Radiology Market

Following up from my prior post on the life of a radiology procedure, I figured I would try to define the radiology market a bit.  Segmenting any market is always tough because there are so many combinations and variations.  My understanding is certainly not complete so there is bound to be errors in what I describe here.  Any feedback on how I can make this more accurate would be appreciated.

Patient Facing Organizations

Luminary Hospitals (~5% of the market)
A luminary hospital is one that is well known in the industry for one or more of the following reasons:

1. Hospitals associated with educational institutions that train radiologists (e.g. Stanford, MGH)
2. Hospitals that are actively engaged in research activities 
3. Hospitals that provide world class service and outcomes for specific procedures or diseases (e.g. MD Anderson Cancer Center, Intermountain Healthcare, Mayo Clinic)

Individuals that work for luminary hospitals are highly sought after by vendors as they provide validation of their strategy and products.  Luminary hospitals tend to lead the industry with respect to innovation and IT solutions.  They often have very strong IT teams and visionary individuals that can apply technology to solve clinical problems (e.g. Dr. Paul Chang, Dr. Paul Nagy, Dr. Eliot Siegel).  The quality of care at luminary hospitals is considered the best you can get.

Mid Level Hospital (~20% of the market)
A mid level hospital is one that is not considered a luminary but can be quite large and provide specialty services.  There is a wide range of IT sophistication at the mid level hospitals.  Some have internal IT teams while others will outsource the entire IT team to another company such caretech.  The actual quality of care received from mid level hospitals typically ranges from good to excellent.

Community Hospital (~30% of the market)
A community hospital is what you find in smaller cities and rural regions.  Community hospitals are quite small, provide limited services and tend to provide below average quality of care.  The level of IT sophistication is generally quite low.

Imaging Centers (~20% of market)
Imaging centers are organizations focused on radiology imaging only.  They typically own their own equipment and have their own radiologists read the studies.  They generally charge less than hospitals and range in quality.  They often provide services at or below a mid level hospital.  Imaging Centers are highly focused on reducing operational costs and tend to view products from that perspective.  The largest Imaging Center is RadNet, you can find a list of them here.

VA (~10% market)
The VA takes care of US veterans and have hospitals throughout the US.  They are organized into regions called a "VISN".  See a map of the VISN's here.  One interesting thing about VA hospitals is they were the first in the USA to have a true EMR which is called VISTA and open source.

Department of Defense (~5% market)

The Department of Defense (or DOD) provides healthcare for government officials and active service personnel.  They have very high security standards which vendors must pass before they can sell to the DOD.  

Other Related Organizations

Private Practice Radiology Groups 

A private practice radiology group is a company that provides diagnostic interpretation for any of the above hospitals.  Radiology groups range in size from an individual radiologist to a group of 50+ radiologists.  Radiology groups typically perform reading services for multiple hospitals.  You can find a list of the top private practice radiology groups here.

Teleradiology Services
Teleradiology services came in to being to provide after hours reading for hospitals.  Smaller radiology departments have trouble covering late night or weekend reads and contract with a teleradiology service to do this for them.  The largest teleradiology service company is vRad

Integrated Delivery Networks
An integrated delivery network (or IDN) is an organization that consists of many hospitals that are integrated to provide more cost effective healthcare.  IDN's have come into being over the past 15 years as midlevel hospitals have struggled to stay profitable.  IDN's are quite large and therefore have large IT groups and high expectations regarding product scalability, availability and integration.  You can find a list of the top IDN's here.

Group Purchase Organizations

Group Purchase Organizations (or GPOs) are organizations that have hospitals as members that group up to get better pricing.  If a vendor can get on the approved list of a GPO, it can make sales much easier at the cost of a lower sales price.  A list of the top GPO's can be found here.

Mobile Providers
A mobile provider has CT and MRI scanners on trucks that can be driven to a given location to increase capacity at a hospital.  The biggest player in mobile MRI is Alliance Healthcare.

The Life of a Radiology Procedure

I figured I would jot down some notes on what I have learned about radiology procedures.  I am doing this in hopes that others will find it useful and also that some may correct my understanding.

The Payor
To start off with, it helps to understand where the money comes from to pay for radiology procedures.  The term "payor" is often used here and can refer to one of many things:
1) Health insurance company (e.g. United Health Group, Wellpoint Inc Group, etc)
2) Health Maintenance Organizations (e.g. Kaiser Permanente, CIGNA, Sharp Health Plan, etc)
3) Private Insurance (companies with self insurance plans)
4) Government (Medicare, Medicaid, VA)
5) The patient (uninsured, HSA)

While the US does not have a single payor like other countries do - Medicare is by far the biggest and tends to set prices across the industry.  When medicare decides to cover a new procedure, almost every other payor tends to as well. Likewise, if medicare changes the reimbursement rate for a procedure, the other payors adjust accordingly to.

Ordering the Exam
The origin of imaging procedure begins with the physician that deems it is necessary.  In most cases, a physician must justify why they think the procedure is necessary.  These justifications are often called "conditions".  For example, if your child falls off the monkey bars at school and hurts their wrist, you would take them to a physician and they would suspect the wrist is broken.  The doctor would place the order for an xray of the wrist with a condition of "acute wrist trauma".  Some payors have electronic systems that can verify that the condition is appropriate for the procedure.  For example, ordering an xray of the wrist when the condition is "headache" should be rejected as it is unlikely to help.  Likewise, ordering A CT of the wrist for acute wrist trauma condition would be overkill as an XRay would suffice.  The ACR has defined a set of appropriateness criteria which can be licensed for use in ordering systems.

Assuming the ordered procedure is found to be appropriate, the next step is to confirm authorization of payment.  This step is important because doctors want to get paid and they need to confirm that the payor will in fact pay for it.  In some cases, a patient does not have insurance but claim they do.  Or perhaps they present the insurance card for an old health insurance company that is no longer covering them.  Prior authorization has traditionally been done manually - a person would call or fax the insurance company and wait for a response.  This is of course not very efficient - fortunately this is being automated via some recently defined EDI transactions 278. Informatics In Context has a very nice automated prior authorization system, check it out if you are interested.

Scheduling The Exam

Once the exam has been ordered, it needs to be scheduled.  This requires many things:
1) Finding a device that can do the procedure.  You can't schedule a CT exam on an MRI scanner.  Likewise, you cannot schedule a CT brain perfusion procedure on a 4 slice CT scanner.
2) Finding a time slot where the appropriate medical staff are available.  Some procedures require staff with specific training - for example, contrast injection must be performed by a physician
3) Finding a time slot long enough for the procedure.  Different procedures take different lengths of time.
4) Finding a time slot the patient is available for.

Scheduling is often managed by what is called a RIS or Radiology Information System.

Performing The Exam

Ideally the patient shows up on time, registers at the desk, gets scanned and then goes home.  Unfortunately things don't always go as planned - patients show up late, procedures take longer than expected, critical staff are late or don't show up, the equipment breaks or emergency procedures take precedence.  The images are generally sent using the DICOM protocol to a PACS or Picture Archival and Communication System.

Reading the Exam

Once the exam has been performed, a radiologist reviews it and issues a report.  The report includes their findings (positive and negative).  The radiologist typically uses a PACS workstation to read the images.  A PACS workstation is usually a Windows based PC with two or more high resolution grayscale monitors.  Here is an example of a radiology report.

Billing the Exam

Each radiology procedure is billed in two parts - the technical fee (performing the exam) and the professional fee (reading the exam).  Each of these parts has a CPT Code.  Payors will reimburse CPT codes at different rates depending upon where the exam was performed and where it was read.  For example, an imaging center would typically be reimbursed less than a hospital.  Likewise, one imaging center may get reimbursed at a higher rate than another imaging center for the exact same procedure.  The reimbursement rate can vary significantly - for example, an imaging center may only be reimbursed $600 for a MRI procedure but a hospital across the street could get reimbursed for $2000 for the sam procedure.  If you are paying for imaging out of your own pocket (no insurance or HSA), it is wise to shop around.

In some cases, the billing is split between two organizations.  The organization that performs the exam would bill the technical fee and the organization that reads the exam bills the professional fee.

Reviewing the Exam

Once the exam has been read by the radiologist and finalized, it is made available to the ordering physician and possible the patient. The ordering physician is mainly interested in the "impressions" part of the report.  This part is a quick summary of what was found and what the radiologist recommends next.  Ordering physicians may want to look at the images and/or ask the radiologist a question about the report.

Development Software

As a follow-up to my prior post, I figured I would share some of the tools I find useful for writing software.  Hopefully I didn't misuse any of these memes too much:

JetBrains WebStorm.  This is the IDE I use for developing cornerstone and other web applications.  I had been using Visual Studio for web development but I found it lacking in many ways and of course doesn't run on Mac OS X.  I also tried Eclipse and Sublime but found them inferior to WebStorm.  JetBrains has some amazing engineers - you really can't go wrong with any of their products.

Visual Studio 2013.  Visual Studio has been my primary development environment for most of my career so I know it very well.  When it comes to Windows development (C# or C++), it is by far the best choice.  It is generally considered one of the best IDE's (if not the best) out there.

ReSharper.  If you write C# or .net code and don't use this - stop what you are doing right now and go download it.  It will improve your code quality and productivity significantly.  Go download it.  Now.  Then buy me a beer the next time you see me.

Google Chrome.  I do most of my web debugging (and normal web surfing) using Google Chrome.  It has great debug tools built in and works well.  Before chrome I used Firefox + Firebug as my main development browser but it would get really slow during debugging at times - I don't have that problem with chrome and don't have to install anything extra - chrome's built in tools are as good as firebug for the most part.  I don't miss firebug at all.  I only use IE11 when I absolutely have to - its debugger is terrible which is a huge disappointment - Microsoft is clearly trying but just not there yet.

Node.js.  Node.js is best know for writing server side logic using JavaScript.  What it is not as well known for is writing automation scripts for builds and other tasks.  Using Node.js is really nice because I can re-use my JavaScript expertise for doing these tasks rather than yet another language.  There are several nice build automation tools that are built on Node.js - I have used grunt with great success but I have also heard that gulp is even better - some day I'll play around with it.  If you are still using powershell, shell scripts, python or any other language for scripting - you should definitely give Node.js a try

git.  Git is the best SCM out there hands down.  If you are using something else - take the time to learn it.  We use github to host our open source projects and private repositories and it is fantastic.

DevPartner Bounds Checker.  If you are writing C++ code you will eventually run into memory corruption bugs.  This doesn't happen to me that often but when it does, Bounds Checker is the tool to use.  They have a $149 99 day edition which is great for when you have to get your hands dirty on some old legacy C++ code.

Notepad++.  Far better that notepad on Windows and has some cool JSON plugins that are very useful.

XCode.  I wrote most of the 3D render server in XCode just to see what it would be like.  Overall it wasn't too bad - while I still prefer Visual Studio for C++ development, XCode has some nice features related to performance tuning built in that Visual Studio does not.

7-Zip.  Simple, fast and compresses well. 

My New Development Setup

I have had people ask me about my development setup and since I just upgraded to a new machine this past weekend, I figured I would blog (*cough* brag *cough*) about it.  First a little history:

I have been a Windows developer for most of my career so my work development machines have always been Windows based.  I switched my home computer to Mac in 2005 and can honestly say my life is better because of it.  I have 3 kids and a wife and have had to do very little system admin work at home during the past 9 years - Apple stuff just works.  When I joined OnPoint Medical Diagnostics in 2011, I decided to switch to Mac for work even though we were developing on Windows.  This may sound strange to some, but Apple hardware is generally cutting edge and the unibody MacBook Pros are incredibly sturdy.  I have heard some Microsoft people say that Windows runs best on Apple hardware - go figure.

I initially ran Windows via vmWare fusion on a  bootcamp partition and that worked great - I could run Windows in a virtual machine under Mac OS X or reboot into native windows if I really wanted the speed.  I quickly discovered that I didn't need the extra speed so I ended up running Windows under vmWare on Mac OS X for several years without any issues.  In fact, I found this setup had many advantages:

1. I put non work stuff like iTunes, Spotify, gmail, web surfing and games on Mac OS X and kept the Windows VM purely work stuff (visual studio, source code, outlook)
2. It is also nice to be able to surf the web in Mac OS X while Windows updates are being installed (which seem to take several minutes and happen every other week).
3. You can test (and debug) web apps on iPhone and iPad simulators via XCode

Last April I started development of a 3D Visualization Server to integrate with cornerstone.  The server is written in C++ and uses the excellent VTK open source project.  VTK features the ability to do volume rendering using a software renderer as well as a GPU renderer.  The software renderer was working great, but I was curious how much faster it would be with a GPU.  Unfortunately my Mac Mini at home didn't have a GPU so I couldn't test it - I ended it up getting a good deal on a mint condition 2012 15" Retina Macbook Pro off of craigslist which had a GPU.

The new 2012 MBP was amazingly fast - unfortunately VTK appears to have a bug and the GPU renderer produces black images.  Interestingly enough, the GPU rendering works fine when run on a Windows virtual machine under Mac OS X (GPU rendering is blazing fast - 60 FPS at full image quality).  As time went on, I ended up building a number of virtual machines to support various consulting projects (each customer gets its own VM) and I ran out of space on the MBP's 256 GB SSD drive.  I bought a couple of MyDigitalSSD OTG drives to store the VMs (these are absolutely AWESOME - highly recommended) but I had all these cables laying around and it was disrupting my work area feng shui so I was getting the itch to upgrade again.

The key criteria for a new machine was the following:
16 GB RAM Minimum, ideally 32 GB RAM
i7 Quad Core
1TB PCIe flash storage
GPU

The only two options were a 15" Retina MBP or a MacPro.  I have actually been wanting a MacPro since they came out but they are pricey and I would still need a laptop for travel which adds even more cost.  Two separate machines also means I have to deal with data synchronization and aint nobody got time for that.  It turns out that the 15" MBP is actually faster than than the least expensive MacPro - check out the performance comparisons here.  The only issue I had with the MBP route is that the maximum RAM is 16GB which tends to disappear quickly when you have a few VMs running.  This past weekend I decided to bite the bullet and picked up a top of the line 15" MBP and absolutely love it.  It feels about 33% faster than my 2012 MBP - most of which I am guessing is from the PCIe flash drive.  Check out these numbers:

In addition to the MBP, I have an Apple Thunderbolt display, old Dell 24" monitor in portrait mode and iPad air all on top of a nice glass desk.  I use an Apple keyboard with number pad and a Logitech g500 mouse.  The MBP sits in a vertical stand which helps keep it cooler.  Here is a picture of my setup:

Amazon AppStream - the savior for server side rendering?

Amazon AppStream provides a platform for server side rendering that promises to be interactive enough for gaming. Since gamers have high standards for interactivity, this technology could very well enable cloud based server side rendering for medical imaging.

Currently AppStream only has servers in the US East region of AWS (located in North Virginia) which means that interactivity will decrease the farther you get from the datacenter. Amazon does plan to expand AppStream to the other regions which are located in Northern California and Oregon in the US (click here for zones outside of the US). The system will automatically connect clients to the server that will provide the best experience which is really convenient for application developers. Interactivity may indeed be very good for the west coast and some parts of the east coast, but it may not be good enough for the rest of the country. Amazon could of course solve this problem by adding additional AppStream data centers across the country and this seems reasonable if the service is indeed successful.

One cool trick AppStream uses is H.264 to stream the screen from the server to the client. H.264 is one of the more popular video codecs in use today - it is used in blu-ray discs, iTunes and youtube. H.264 is computationally expensive to encode so using it for real time streaming is truly impressive. They must be doing the encoding using the GPU or perhaps even specialized hardware.

While this technology certainly has potential, the lack of encryption of the video stream will prevent it from being used for PHI due to HIPAA.  Life Sciences and medical imaging is a stated use case so hopefully they can resolve this in the future.

Note: All of the above information is based on what I gleamed from the publicly available information on the Amazon AppStream web site (specifically the FAQs)

Server Side Rendering

The term "Server Side Rendering" in medical imaging generally refers to executing all or a portion of the image rendering pipeline on a server and sending the resulting image to the client.  There are many benefits of server side rendering:

1. A server can usually load image data from the archive or cache much faster than the client can.  Servers are located in the data center which can have 1Gb/s+ bandwidth between each other.  A client's bandwidth will often be much lower especially if it is not in the same physical building.  In many cases a client can have at least 15 Mb/s which is still 66x slower than the datacenter.  To put this into perspective, a 1000 slice CT volume would load in ~ 4 seconds over a 1 GB/s connection but would take over 4 minutes over 15 MB/s.  The faster load time is perhaps the best reason to do server side rendering for volumetric data sets.

2. The result image is typically much smaller than the source image.  For example, a 1000 slice CT volume is 500 MB of data, yet a MPR or Volume Rendered view of that data in JPEG may only be 5 kB in size.  Another example is a 2D projection radiograph is often 30 MB in size, yet a rendered JPEG of that image may only be 30 kB in size.

3. Minimal client side hardware dependencies.  Volume rendering and MPR often require higher end hardware than is often found on client machines.  For example, a GPU based rendering engine often requires specific GPU cards and drivers to function which are not usually found on enterprise desktop PCs.
4. Minimal client side software installation.  Medical imaging software can be difficult to install to the enterprise due to the variety of operating systems and hardware found on enterprise PCs.  One strategy to deal with this is to install the software on a machine in the datacenter and allow a client to access it via a remote desktop protocol like VNC or RDP.

While server side rendering sounds great, interactively suffers dramatically when latency is present.  While bandwidth is generally not an issue for medical imaging today, latency still is.  If you think about how server side rendering works, the client has to send user activity (mouse, keyboard presses) to the server where it gets processed and then sends back the resulting image.  The time it takes to send a message from the client to the server and then back again is called latency.  In the enterprise, latency is usually quite low - around 1 ms.  Low latency like this provides a highly responsive user interface that many users cannot distinguish between local client side rendering.  Unfortunately many users need to access images outside of the enterprise where latency quickly becomes an issue.  Assuming server side rendering is instance (which is not necessarily the case), you get the following latency to best case frame rates:

25 ms ping = 40 FPS
50 ms ping = 20 FPS
100 ms ping = 10 FPS

In a MAN or WAN you may have 25-50ms which will meet most users needs.  Once you get outside of the MAN/WAN (e.g. across the country or planet), latency quickly jumps up to the 80-150 ms range.  This is a major issue for remote users as well as any cloud based system that uses server side rendering.  The major cloud vendors such as Amazon and Azure do have a number of data centers that are geographically located throughout the country (and world).  It is possible to deploy a server to each datacenter and connect the client to the server with the lowest ping times.  This doesn't happen automatically though and is something to consider when looking at any cloud based solution.

Simple C++ GZip codec with Zlib

I needed a simple gzip compress/uncompress function but was having trouble finding a good example.  I am posting this very simple C++ GZip codec using Zlib in hopes that helps others:

https://github.com/chafey/GZipCodec

PS - I discovered boost has gzip support after I coded this, definitely use that instead of my code if you can.

WADO-RS Overview

WADO-RS was recently added to the DICOM standard in 2011 with Supplement 161.  The RS stands for REST or RESTful and is generally easier to understand and work with than WS* Web Services.  WADO-RS was mainly driven by the need to provide a way for clients to access multiple SOP Instances in one HTTP request which had been shown by the MINT project to offer significant performance gains.

One of the key concepts that WG27 took from the MINT project was the concept of bulk data.  A bulk data item is a field in a DICOM SOP Instance that is typically very large - such as the Pixel Data field 7FE0,0010.  To maximize performance, all fields for a study can be retrieved at once but bulk data fields are replaced with a URL that can be used to obtain the bulk data item via a separate request.  This strategy enables clients to stream the pieces of the study they want, when they need them.  This strategy is often used by image viewers to deliver images "on demand".  

WADO-RS provides multiple ways to access SOP Instances to support a variety of use cases and scenarios:

1. RS – RetrieveMetadata.  This allows a client to retrieve all fields (except bulk data) for all SOP Instances in a study.  It supports both XML and JSON responses.  The JSON response is an array of objects, each of which contains all of the fields for each SOP instance in the study.  The data in bulk data fields is replaced with a URL which can be used to get the actual bulk data separately.  The XML response is a multi-part MIME message with each SOP Instance returned as a separate XML document and encoded as a single part.
2. RS – RetrieveBulkdata.  This is the mechanism to retrieve a single bulk data item as returned in the RS-RetrieveMetadata response.  By default the bulk data is returned in little endian transfer syntax, but other transfer syntaxes can be requested (e.g. JPEG2000)
3. RS - RetrieveFrames.  This mechanism allows a client to get all image frames for a study, series or SOP Instance in one request.  The frames are returned in a multi-part mime message with each frame encoded as single part.  By default frames are returned in little endian transfer syntaxes but other transfer syntaxes can be requested (e.g. JPEG 2000).  
4. RS – RetrieveStudy.  This allows a client to obtain all SOP Instances for a study in one request.  Each SOP Instances is sent as a separate part in a multi-part MIME message with each SOP Instances as a DICOM P10 byte stream (application/dicom) encoded in a single part.  You can also request just the bulk data items for a study and they are returned in a multi-part MIME message with each bulk data item as an individual part.
5. RS – RetrieveSeries.  Same as RS-RetrieveStudy but scoped to a series.
6. RS – RetrieveInstance.  Same as RS-RetrieveStudy but scoped to an individual SOP Instance

DICOM WADO and WADO-URI

The DICOM standard defines web service based functionality in PS 3.18.  DICOM Working Group 27 (Web Technology for DICOM) oversees the standardization of web services.  You can read more about WG 27 in the DICOM's strategy document and access its meeting notes.  

The first web service mechanism standardized was called WADO (Web Access to DICOM Objects) which provides HTTP GET access to SOP Instances.  WADO was added via Supplement 85 in 2003.  WADO was recently renamed WADO-URI to avoid confusion with the new WADO-RS (RESTful) and WADO-WS (WS*) standards.  This industry is still coming up to speed with this new terminology, so the term "WADO" generally refers to what is now called WADO-URI as WADO-RS and WADO-WS are not widely in use yet.  

WADO-URI supports access to SOP Instances via HTTP GET.  Built into HTTP is a mechanism to request data in a variety of formats (or MIME types).  There are hundreds of MIME types defined, here are some of the more common MIME types that are behind the web pages you see when browsing the web:

• text/html - HTML documents
• text/css - CSS documents
• image/jpeg - JPEG images
• image/png - PNG images

Different types of SOP Instances can be returned as different MIME types.  For example, a single frame image SOP Instance can be rendered as a image/jpeg but multi-frame image and structured report SOP Instances cannot.  A structured report can be retrieved as an HTML document, but an image SOP Instance cannot. 

DICOM also defined its own MIME type 'application/dicom' which refers to a DICOM P10 byte stream.  By default, a requested SOP Instance should be returned in Explicit VR Little Endian transfer syntax.  Other transfer syntaxes can be requested (e.g. JPEG 2000), but the server does not have to honor the request.

Here is an example of a WADO-URI URL that requests a SOP Instance rendered as a JPEG:

http://localhost:8080/wado?requestType=WADO&studyUID=1.3.6.1.4.1.25403.166563008443.5076.20120418075541.1&seriesUID=1.3.6.1.4.1.25403.166563008443.5076.20120418075541.2&objectUID=1.3.6.1.4.1.25403.166563008443.5076.20120418075557.1

A few notes about this URL:

1. You must know the StudyInstanceUID, SeriesInstanceUID and SOPInstanceUID.  The QIDO-RS standard will allow you to query for these via a REST call, but it is new and very few systems that support it right now.  
2. The default MIME type is application/jpeg which returns a JPEG image the same size as the DICOM image with a server selected window/level value.  Your web browser will display this rendered image if you paste the URL into it!

While rendered images are nice for integration with web based applications, there are three disadvantages:

1. You need to re-render the image on the server if you need to adjust the window width or window center
2. You don't have access to any other DICOM Fields (e.g. patient name, patient id, study description, etc)
3. The image is not diagnostic due to encoding as a JPEG Lossy image (this can be avoided if the server supports rendering to the image/png MIME type)

To address these issues, you can request the SOP Instance as a DICOM P10 byte stream.  To do this, you need to request that it be returned with the application/dicom MIME type using the contentType parameter.  Here is an example of this for the same SOP Instance used above:

http://localhost:8080/wado?requestType=WADO&studyUID=1.3.6.1.4.1.25403.166563008443.5076.20120418075541.1&seriesUID=1.3.6.1.4.1.25403.166563008443.5076.20120418075541.2&objectUID=1.3.6.1.4.1.25403.166563008443.5076.20120418075557.1&contentType=application%2Fdicom

Note that the '/' in 'application/dicom' is converted into %2F due to conform with URL encoding rules.  Diagnostic viewers will typically use the application/dicom access method to avoid the three issues listed above with the image rendered mode.  One drawback to the application/dicom access method is that the responses are typically much larger.  An uncompressed 256x256 MRI image is 128K bytes while a JPEG rendered version may only be 3k!   

Working around CORS

While building web applications, I sometimes run into the case where my http requests fail because the web server does not support Cross-origin resource sharing or CORS.  One way to workaround this by using a http proxy which adds the missing CORS header to the response.  This can be easily done with Node.js and the http-proxy library.  Here is the script that will create a proxy that listens on port 8000, proxies each HTTP request to localhost:8042 and return the response with the CORS header added:

var http = require('http'),
    httpProxy = require('http-proxy');

var proxy =  httpProxy.createProxyServer({target:'http://localhost:8042'}).listen(8000);

proxy.on('proxyRes', function(proxyReq, req, res, options) {
  // add the CORS header to the response
  res.setHeader('Access-Control-Allow-Origin', '*');
});

proxy.on('error', function(e) {
  // suppress errors
});

You can also workaround this by disabling CORS in your browser.  For example, chrome can be started with the --disable-web-security flag.

Image Rendering Refactoring

While working on the faster image loading mechanism for Cornerstone, I found myself making further changes to the drawImage() function.  I already felt this function was too complex as it had to deal with at least 12 different flows due to the different types of images and caching scenarios.  I realized that adding support for the different faster image loading the function would quickly become hard to read and unmaintainable.  Clearly something had to change, but what?

After thinking about it for a bit, I had an "aha" moment and realized that the image rendering responsibility should be moved to the image loader.  The image loader design was already providing flexibility with respect to the image format and protocol used and it would also have to be aware of the various fast image loading techniques being used.  Adding the image rendering responsibility to the image loader will allow the fastest possible rendering.

The only issue with moving rendering to the image loader is code reuse.  I envision a wide variety of image loaders that pull full uncompressed pixel data from different servers in different formats.  For these types of image loaders, it doesn't make sense to have them cut and paste the rendering code from another image loader.  It would be much better to have this generic image rendering mechanism in a shared location.

Based on the above, I decided to split the drawImage() function into three new functions - renderGrayscaleImage(), renderColorImage() and renderWebImage().  This simple refactoring immediately made the code easier to understand so I knew I was on the right path.  The drawImage() function was simplified to some boilerplate logic and delegated the actual rendering to the image object returned by the image loader.  I then proceeded to modify each of the image loaders to call one of the three newly created render functions and everything was working again.

This simple refactoring not only made the existing code simpler and easier to understand, but it also prepares Cornerstone for the more complex functionality that will be needed to handle faster image loading.  You can find the commit for these changes here

Implementing a QIDO-RS Service

Having previously implemented a basic QIDO-RS worklist using JavaScript, I figured it would be useful to implement a QIDO-RS service to improve my understanding of the standard. For those unfamiliar with QIDO-RS, it allows web based queries for studies, series and instances - similar to C-FIND. This weekend I implemented a simple service implementation using C#, Visual Studio 2013 and ASP.NET MVC 5 WebApi. You can find the source code for this project on my github here.  

Overall I found the implementation to be fairly straightforward and it took me about 16 hours total. About 2/3 of this time was related to a) understanding the standard and b) working through various issues with Visual Studio 2013 and ASP.NET MVC 5 WebApi (neither of which I had used before).  

Here are some of my thoughts after getting this implemented:

1. I found several errors in the DICOM standard:

• Modality has incorrect tag in Table 6.7.1-2a
• Missing Study Description tag in Table 6.7.1-2
• Misspelled StudyInstanceUid on line 641
• JSON Example in F.2.1.1.2 is Invalid JSON (missing a comma between array entries)

2. The standard was difficult to understand

• It makes several references to concepts documented elsewhere (e.g. fuzzymatching,)
• It seems to be designed as a web wrapper around CFIND which therefore requires fully understanding CFIND

3. It isn't clear what functionality is required and what is optional. This may be due to it being designed as a wrapper around C-FIND and I don't have expert level knowledge of C-FIND.

4. The JSON mapping was not designed for ease of use by JavaScript developers.  From F.2 in the standard "The DICOM JSON Model follows the Native DICOM Model for XML very closely, so that systems can take 790 advantage of both formats without much retooling".  Here are some specific issues

• The attribute tag (group/element) is used as the property name in the JSON object. JavaScript code cannot use dot notation to access these properties since it is not supported for property names that do not begin with a letter. This could have easily been solved by putting a letter like x in front of the group/element.
• The inclusion of the VR field seems unnecessary since the native JavaScript type system is used.
• Putting all values in an array is awkward. It would have been better to only use an array for attributes with multiplicity > 1
• The use of Alphabetic, Ideographic and Phonetic in the PN mapping is unclear. I am sure these are documented somewhere else in the standard - but where? Having a PN value be an object instead of string is also a bit awkward.
• The casing for property names is not consistent. vr starts with lowercase letter while the others start with an uppercase letter. The most common naming convention in JSON/JavaScript is the first letter lowercase.

Overall this is a positive step forward to bringing DICOM into the world of the web but more can be done to reduce the barriers to using DICOM by developers. The best way DICOM can reduce these barriers is to make web browser and JavaScript based consumption a top priority and make it as easy to use as possible in that environment. Efforts such as the dicomWeb online documentation are very helpful and so are open source implementations of the standard.

Fast Image Loading with Cornerstone

During SIIM 2014, Dr. Paul Chang stopped by to see the Cornerstone demo and reminded me that you not only need fast panning/zooming/window/level but also fast loading of images. So far I have implemented a WADO image loader and plan to eventually build one for WADO-RS.  The problem with these standard protocols is that the entire image needs to be loaded before a useful view of the image can be displayed. While it is possible to draw the image from top to bottom as the pixels are received, users generally find such behavior unacceptable.  As a rule of thumb, users expect a system to respond to user input in less than a second - including the display of images.  This is a real challenge for medical images which are typically lossless, high resolution and have more than 8 bits per pixel.  One or more of the following techniques are often applied to improve the time to first image:

1) Variable Compression. A variety of compression algorithms exist each with different characteristics for speed, size of bitstream and image quality. One can imagine using an algorithm that is fast, produces small bitstreams and has low image quality to display an initial view of the image and then followup with another view that is generated from another algorithm that is slower, produces a larger bit stream and has higher image quality. This is exactly how the cornerstone 3D image server works - it sends a lossy JPEG when the user is interacting with the system and then follows up with a lossless PNG once they stop.

2) Image Tiling. If you have used google maps, you have seen tiled rendering at work. A large image is broken up into smaller pieces called tiles at different resolutions.  When the image is first displayed, a very low resolution tile is retrieved and then scaled to the viewport size.  This results in a "blurry" image but is then replaced with a sharper image as the higher resolution tiles are retrieved.  As the user pans the image, regions of the image are exposed that do not have high resolution tiles.  These regions are again displayed by scaling up the lower resolution tile and sharpened once the higher resolution tiles are retrieved.

3) Interlaced or progressive encoding. Image pixels can be encoded such that a reasonable view of the image can be displayed after reading just a portion of the bitstream and then updated to show the entire image after all bits are read. The simplest way to do this is to interlace the image - encode all the even rows first followed by the odd rows.  In this case, a reasonable view of the image can be displayed after reading half of the bitstream as the system can make up the missing odd rows by interpolating between the even rows. Once the remaining bits are read, the interpolated odd rows are replaced with the real data and the image is updated. More complex version of this is the Adam7 Algorithm (shown below) or a Discrete Wavelet Transform.  Note that standard web image formats like PNG, JPEG and GIF all have support for some form of interlaced or progressive encoding but do it slightly differently.

4) Server side rendering.  In this case, the user input is sent to the server side to process and a rendered image is produced and sent back to the client for display.  Remote screen sharing technology such as VNC or Remote Desktop are perhaps the simplest form of server side rendering.  More advanced forms exist by having the client interpret the user input and then make calls to the render server accordingly. This strategy works very well on an enterprise network where latency is low but quickly becomes unusable as you move outside the enterprise (e.g. from other side of city, home or across the country).  When it comes to interactivity, most users find < 10 FPS unusable.  To achieve 10 FPS, each client/server response must complete in less than 100 ms which includes network latency and render time.  Outside the enterprise network, latency starts around 40ms and goes up as you get farther away leaving little time for the server to actually handle the request.  Due to the sensitivity to latency of server side rendering, it will be less attractive when compared to solutions that use client side rendering where the only limit is the available processing power.

All of these techniques result in changes to the underlying pixel data for an image. Currently Cornerstone is designed around static images - that is images where the pixel data does not change. To support the techniques listed above, the following is required:

1) It must be possible for a region of the pixel data to be externally invalidated when the underlying pixel data has been updated for that region. This invalidation will cause Cornerstone to re-render the image so the changed pixels are displayed to the user
2) It must be possible to detect when the user interacts with the image in a way that requires changes to the underlying pixel data. Cornerstone currently emits the CornerstoneImageRendered event when the image is rendered which can be used to detect changes to translation, scale and viewport size but it there may be better ways to deal with this.
3) It must be possible for Cornerstone to render subviews of an image. Currently cornerstone always renders the entire image even if all the pixels aren't displayed. This strategy doesn't work at all for very large images like pathology slides where the client doesn't have enough memory to hold the entire slide in the first place. An interesting side effect of this change is that existing operations like WW/WC might get faster for images that are larger than the viewport as it would only need to push the viewable pixels through the LUT for each frame (rather than all pixels like it does today).

While implementing this is quite challenging - the bigger challenge is how to implement it while keeping Cornerstone easy to understand.  I am just now beginning to think through the design for this and expect it to take me several iterations to get right.  Look for future posts about this to see how the design emerges.