Birthing a Book...

I'm getting down to the final writing and editing process before release of the
"Wireshark Network Analysis" book (the name is still being debated)... low on
sleep... high on ideas...

In my experience, there are two ways to write a book – “carve it” or “birth it.”

When you carve a book, you start with a large piece of granite and painstakingly
shape each part of the book through laborious research on topics you do not
deal with on a daily basis, but you must cover in the tome. The chisel you use is
the size of a toothpick and at times it feels just as fragile against the cold,
unfeeling slab of rock you have decided to mold.

Birthing a book is easier than birthing a human child...

When you birth a book, the entire book is formulating in your head – the table of
contents is taking shape, the tone and depth is coming into focus until… plop!
There it is! You just need to get it out of your head in a clear and concise manner,
get it formatted, proofread and bundled in a warm blanket to show to the world.
The "Wireshark Network Analysis" book has been written by the "birthing"
method.

In essence, I was in labor for 20 years – and yes, damn it, I’d
like a bit of sympathy!

Twenty years ago I presented a session on ARCnet communications to a group
of peer instructors. I delved into the idea of packet structure and the mythical
belief at that time that everyone cared. Somehow though, I related the ARCnet
networking rules and limitations to Sister Gerald, the militant no-nonsense head
of discipline at my Catholic boarding school… and I got a few laughs.

Imagine that… networking can be funny!

(BTW... my spell checker is freaking out over the word "ARCnet"! Ha ha...)

Now – before you think I’m going to mention any of the nuns, my techno-
challenged father, my WoW-addicted son (go Alliance!), my iPhone toting
daughter (who I hope will grow up and make iTunes a less pathetic application)
and my Pavlovian response to a trace file filled with hideous communications
issues and delicious security flaws – this book is not a breezy stroll through the
world of packets.

This book is packed with basic through advanced techniques, tips and tricks to
analyze a variety of network types. It is designed to get you from point A to point Z
(or perhaps I should say point 0x00 to point 0xFF) as fast as possible with a
solid understanding of the processes, protocols, and putrid things that occur
under our noses (or under our feet).

If you don’t have Wireshark loaded on every computer within reach, stop now!

Wireshark is the best girlfriend/boyfriend, wife/husband, mother/father,
sister/brother, dog/cat or lover you have ever had.

* Who is always there to listen to you with a patient and understanding
silence when you are crying in your latte because the users keep
complaining about network performance? Wireshark.


* Who never threatens to fire you if you don’t get those file transfers to occur
at ‘acceptable speeds’ before lunch today? Wireshark.


* Who smiles and sits around all day long just waiting for the moment you
say “I need help”? Wireshark.


That’s right.

So… it’s time to elevate Wireshark from “network wallflower” to network
powerhouse. It’s time to roll up your sleeves(1), get rid of the training wheels, put
on your helmet and reflective gear, tell everyone to get the hell out of your way, get
on that bike – and ride!

By the way – you have no idea how difficult it was to refrain from adding humor
(or at least what I call humor) to this book. It crept in at various points – some I
left in (nestled inside tips throughout the book). Most humorous comments I
simply moved aside for a later book that might focus on the funny side of packet
analysis. I’ll have to wait – that one isn’t even in the early gestational period yet.

Happy New Year to all.

Enjoy life one bit at a time!
Laura

(1) I recently heard a talk show host state that "In difficult times, there are those
who throw up their hands and others who roll up their sleeves"... I am likely the
latter.

Strange SYN Scans and SpongeBob Slippers

Nestled in my trusty Sponge Bob Slippers and surrounded by mounds of tissues boxes, cough syrup with Codene, a temperamental foot heater, NetFlix on-demand and business cards of every Chinese restaurant in the neighborhood... I settled in to study signatures of Nmap's fascinating OS detection process. Ahhh.... comfort packets....

First - if you haven't picked up Fyodor's Nmap Network Scanning book - put that on the top of your to-do list (wait... squeeze "Sign up for Summit 09" just above that). You'll want to snuggle up with pages 177 -178. You can get the book at Amazon or try to reach Fyodor over at insecure.org - buy it directly and ask him to sign your copy - this is a hot book!

Here's the scoop - capture your traffic as you run nmap -sV -O -v against your target (version scanning, OS fingerprinting and verbose mode). Got permission, right? Good. Read on.

Nmap's OS fingerprinting process contains numerous unique packets - by building a series of butt-ugly color filters you can spot these strange packets easily with relatively low concerns of false positives (if you happen to find these packets being sent by another application you should still be concerned - it's weird behavior).

In looking through the trace file and referencing Nmap Network Scanning, I came up two color filters (both with butt-ugly background colors) that caught the majority of the unique packets generated during the scan.

Filter #1
(tcp.flags == 0x00) || (tcp.options.wscale_val == 10) || (tcp.options.mss_val < flags ="="" urgent_pointer ="="" flags="="0x02" flags="="0x02"> 0)

So shall we break this down a bit?

(tcp.flags == 0x00)
This looks for the null scans - TCP scans that have no TCP flags set.

(tcp.options.wscale_val == 10)
The TCP window scale value equal to 10. Although other TCP handshakes may use this value during the handshake process, it is unusual and listed in the book as one of the scan techniques and verified in the trace file of the OS fingerprinting process.

(tcp.options.mss_val <>
This one is a bit sticky - we're looking in the options section of the TCP header for a maximum segment size value smaller than 1,460. This did cause numerous false positives when I ran it on other trace files. Regardless, I like having this in my color filter because it points out some weird starting MSS starting value. As an option, I considered moving this to another color filter with a slightly lighter background color.

(tcp.flags == 0x29) && tcp.urgent_pointer == 0
This filter looks for the FIN, PSH and URG bits set in packets with the Urgent Pointer field set to 0.

(tcp.flags==0x02 && !frame[42:4] == 00:00:00:00)
Yeah - this is a strange one and brings up a change I'd like to see in Wireshark. This looks for packets with the SYN bit set only and the Acknowledgment Number field set at a non-zero value. So... what's the "frame[42:4]" all about? Well... Wireshark does not recognize the Acknowledgment Number field in the first packet of the handshake process as it doesn't have any use in that packet. I'd still like to see the field in those packets so I can filter on it though. I tried messing around with !tcp.ack==0 but that didn't work.

(tcp.flags==0x02 && tcp.window_size <> 0)
This looks for SYN packets with a small window size value and the window scale factor set to 0. This did hit some false positives in other trace files, but they were all hosts with strangely small window size values anyway - a bit of a concern to me anyway. I considered setting this as a separate color filter and may alter my color filters as I test this against more trace files.

Filter #2
I set another color filter for tcp.window_size < syn="="1 - it was a bit lighter in background. This color filter had lots of false positives in other trace files, but pointed out numerous TCP connections that were using non-optimal starting wndow size values.

Ohhh... look at all the pretty, er I mean ugly colors! This particular Nmap scan sequence screams "Halloween All Year Long!"

Join us at Summit 09 as we investigate other malicious traffic patterns! Register over at www.chappellseminars.com/summit09 and I'll see you December 7th!

Enjoy life one bit at a time!
Laura

SSL/TLS Flawed: Using Wireshark to Decrypt Attack Traces from PhoneFactor

It seemed such a coincidence, I sent out a teaser for a project underway and alluded to the security implications - the project, however, was not related to the SSL/TLS vulnerability that hit the public last Thursday.

How bad is this SSL/TLS vulnerability? Amazingly horrid! Listen Up! (MP3 - 1MB) Click here to download Ron Nutter's interview with Steve Dispensa (or grab the .zip file here) - one of the PhoneFactor guys who demonstrated the vulnerability to a working group of affected vendors and representatives of various standards committees.

Read Up!
Steve Dispensa and Marsh Ray of PhoneFactor wrote an 8-page overview of the issue which is based on the TLS renegotiation process. The figure below shows the basic SSL/TLS handshake process.

In Wireshark, the display filter ssl.record.content_type == 22 extracts SSL/TLS handshake packets.

The document written by Steve and Ray defines the security issues demonstrated against recent Microsoft IIS and Apache httpd versions. In essence, the renegotiate attack method defined is used to inject malicious code into the "secure" connection.

One of the most interesting areas of the document focuses on the use of request splicing in which two HTTP requests are combined. The first request triggers the renegotiation while the second request effectively comments out the first request and overrides it with the malicious one.

Analyze the Attacks Yourself
Download the PhoneFactor document, numerous trace files (including decryption keys), protocol diagrams and details here.

Hint: In Wireshark, disable the Preferences > TCP > Allow Subdissector to Reassemble TCP Streams to view the SSL/TLS handshake more clearly.

Step 1: Get the Traces/Keys
Download and extract the files into a directory called "ugly". (Again - download from here.)

Step 2: Set up SSL with Keys
Private keys to decrypt the traces are in the 'caps' and 'certs' directories. For simplicity sake, I recommend you create a \keys directory and copy all the keys there.To decrypt the client_init_renego.pcap file, I used Preferences > Protocols > SSL and entered the following value:

192.168.80.125,443,http,c:\users\laura\keys\ws01.mogul.test.key

When you have successfully set up decryption, your traffic should indicate HTTP in the protocol column and, if colorization is enabled, the lovely lime-green color of HTTP traffic.

Step 3: Follow the SSL Stream
Once you have applied the decryption, you can right-click on one of the HTTP packets and select Follow SSL Stream to reassemble the traffic as shown below.

In the figure at left we can see the request to GET /evil.html and the x-ignore line for GET /index.html. This process of using the ignore header prefix is described on page 3 of the Renegotiating TLS.pdf document.

Inside the SSL/TLS Handshake - Another "Must Read"
Jeff Moser penned an impressive blog entry entitled "The First Few Milliseconds of an HTTP Connection" which analyzes the handshake process, selection of a cipher suite and use of the RSA algorithm. Read Jeff's blog here.

What's the Solution?

The document written by Marsh Ray and Steve Dispensa paints a pretty gloomy picture of possible remedies.

"There appear to be few silver bullets to address these issues."

Ultimately, the fix will require protocol changes - a laboriously painful process that can have unforeseen consequences related to compatibility problems. The paper forthrightly defines the possibility of 'breaking' as well as backwards-compatible protocol changes. It takes serious 01's to throw that 'breaking' term in there. It's no fun being the bearer of such bad news. What a hassle.

In the meantime, I imagine the efforts to exploit vulnerable SSL/TLS connections is underway - those malicious teams might be working longer hours than the vendor/committee teams focused on a resolution.

Big money is at stake.

Enjoy life one bit at a time!
Laura

Double-sided and Double-dumb Printing

Summit 09 Bonus: All Summit 09 attendees will receive a full licensed copy of NetScanTools Pro - a $249 value.

It's not easy going "green"... I mean - all we wanted to do was print the customized course manuals double-sided. What's the big problem?

WIRESHARK and I NEARLY BARFED!



First let me say I was printing to an HP OfficeJet Pro L7780 All-in-One printer - it supports auto-duplexing. The feature is implemented in the most funky way however! Watching the traffic live while the printer did it's 'dance of duplex' - I was shocked to see the lame process of auto-duplex printing.

I had to wonder if the HP printer group might be interested insome classes on networking.*

Ok... here's how this printer does duplex printing. It's automatic, so you don't have to do the old 'refeed the paper and hope it's in the right order' process. You just select auto-duplex and away it goes. The printer prints the first page and - while still holding the paper by the bottom edges, the console reads "Ink drying - please wait." After a while the paper is then sucked back into the printer (a moment that always makes my heart jump - I've heard the shredding sound of mangled paper too many times). The second side of the paper prints and we all sigh relief - another 2-sided page done!

The trace file showed that the printer creates the pause process by sending a Zero Window packet to the client - in essence "talk to the hand" (isn't that HP's little helper logo?). Each 'drying process' essentially was caused by halted flow of print data from my client to the printer. My client sent Zero Window Probe packets to ask 'What is wrong with you, buddy?" and the printer kept the traffic at bay by sending Zero Window Probe ACKs for about 30 seconds.

The printer's TCP window size did decrease down to an unacceptable zero so it wasn't lying about not having any receive buffers. It seems the printer doesn't clear out the buffer space consistently during the print process - it's buffer becomes depleted right at the point when the 'ink needs to dry" - perhaps it knows we'll all need to take a breath then so allowing the buffer to fill doesn't seem like a big deal - it's going to have to sit around idle anyway (anyone think about adding a nice 'ink fan' attachment to make my printing go faster?).

I decided to add a tcp.window_size column and filter on traffic coming from the printer. Using File > Export > File and choosing the displayed packets, I created a csv file with a column just depicting the TCP receive window sizes advertised by the printer. In Excel I selected only the tcp.win_size column and inserted a line graph.

BEAUTI-FRIGGIN-FUL!

Look at the intentional pattern of the window size values advertised with definite zero window conditions during the ink drying time. There's just got to be a better way!

What? Is my printer totally dumb? I didn't think so, but this is certainly dumb behavior. The file is only 13,296 KB! The printer has 64 MB memory standard. Why didn't the printer allow me to send the entire document to the printer, then buffer the data while the ink dried? Why did I have to sit around and wait for the ink to dry before being able to send the next page of data? The Wireshark summary of the data sent to the printer showed the transmission rate averaged around 0.431 Mbit/second - snooze...

I also look at the spooled file in my Windows\System32\spool\PRINTERS directory - holy schmoley! The spooled file (.spl) was 670,016 KB! Whazzup with that? The idea of researching the MS spooling process and .spl file format made my stomach turn again - but it did look like this Powerpoint Notes document was processed as a graphics file.

Then came the really interesting part - I printed twenty-five copies of the custom student manual. You'd think the printer would have buffered the first copy in memory and just pulled from that when it needed to make each successive copy - right? Nope. My client sent the entire file to the printer a second time, a third time, a fourth time... etc. Whoa... that's one dumb process! No turning off my system until the entire job was sent 25 times from the spool\PRINTERS directory. Barf.

Basic network analysis should be a mandatory requirement for any vendor making network-capable devices/applications - or else they might make a network-incapable devices/applications - oh wait - they do!



Enjoy life one bit at a time!
Laura

*Although this is the lamest way to get the job done, I swear by my HP printers - they take a beating and keep on spitting out surreal amounts of pages every month - I am a tactile, visual person - there's nothing like that printed version of the spec to snuggle up with at night! So much for going green. (The only green that will fly around here will be some big $ for printers that can buffer the entire doc once - without the 'talk to the hand process' - and then print extra copies from that!)

Tracing the Route

Summit 09 Bonus: All Summit 09 attendees will receive a full licensed copy of NetScanTools Pro - a $249 value.

During troubleshooting processes, a standard ping test is often used to check connectivity to a host and determine the round trip latency time. This process uses an ICMP Type 8 Echo Request and relies on an ICMP Type 0 Echo Reply.

Sometimes, however, the target won't respond to ICMP Echo Replies - either because it is configured to ignore these ICMP Echo Requests or because a device along the path filters these packets out so they don't reach the target.

I prefer to perform traceroute using NetScanTools' TCP option. Besides setting the TCP port and the sequence number settings, you can also set the MTU (Maximum Transmission Unit) to test the maximum packet size along a path.Another option available is to set the TCP window size - in our example I have set the window size field value to 16,384. In addition, you can define the payload - using a binary or text file.

Why would you use a big fat file for the test? Ahhh... my padawan - to test the MTU allowed through the path and consider putting a signature in the payload that should trigger an IDS or be logged by a firewall - multiple birds with one stone - connectivity testing, latency testing, IDS/firewall testing! Nice!

In the figure below, you can see my host 192.168.0.113 sending a series of TCP SYN packets - the target port is 79 (finger). The packets colored with a red background have an IP header Time-to-Live value less than 5 - a sure sign of a traceroute operation.

When we reach the target, a RST is generated in response. That's what gives us our round trip latency time.I appreciate why companies restrict ICMP-based traffic on their networks - and when I'm doing connectivity tests and latency tests, customizing my TCP-based traceroutes always sits on the top of my to-do list.

Enjoy life one bit at a time!
Laura

Storms Rip the Net

Summit 09 Bonus: Licensed NetScanTools Pro - All Summit 09 attendees will receive a full licensed copy of NetScanTools Pro – a $249 value.

I recognized the tone in the voice that day - the panicked sound of someone dealing with a non-functional network. In this case, the network was a critical one (I can't disclose the specific type of network or customer).

At approximately 3:34am, their critical network came crashing down - no connectivity for any hosts on the network. They'd placed Wireshark on the network and it too crashed within moments.

Ok... so there was something definitely cruising along the network wreaking havoc. I had to see those packets!With over 2,000 miles separating us, it would be a 'walk through the capture' process.

The first step - dump the GUI!

Wireshark comes with Tshark for command-line capture. The syntax used was:

tshark -c 100 -w gen1.pcap

The -c parameter indicates the number of packets to capture. The -w parameter is used to define the name of the trace file to create. Why only 100 packets? What? Well... if there is a catastrophic issue on the network that could kill systems that connect to it that quickly, it shouldn't take many packets to characterize that traffic.

Immediately upon capturing these 100 packets, I instructed the customer to disconnect from the network. You don't need network access to analyze captured traffic - trace files are processed through the Wiretap Library - directly off the disk.

The 100 packets told the story - an insane looping packet storming through the network at a blazingly rapid packets per second rate. When facing a traffic issue like this it is important to look at the IP header's Identification value. You need to differentiate between a looping packet or a series of individual packets sent from a 'killer host' (and I mean killer as in "network killer").

If the IP Identification field value is the same for all the packets, then the packet is looping somehow. If the packet has a different IP Identification field then the packets have each passed through the IP protocol separately from a host. It's an amazingly simple differentiation - and an important one.If the packets had unique IP Identification field values, we'd be looking at a single host causing the problems. We'd be delving into the MAC header of the packet to identify the location of the lousy host. (Having a master list of MAC addresses for all hosts on the network is imperative in that situation. Mark that down as something to do this week!)

In this case, all the IP Identification field contained the same value - this was a looping packet. We had an infrastructure issue. On this heavily switched network it seemed spanning tree was not doing its job. Poor spanning tree - no one really pays attention to it until it has a problem.

Being remote to the customer location, I could not look over their shoulder as they walked the network and shut down one switch at a time. It was in their hands now. I sat waiting for their call - waiting to hear if they'd found the culprit. I didn't wait long.

I waited 30 looooong minutes for the call even though hit had taken the client less than 5 minutes to find a switch that was looping traffic back through the network. They spent the other 25 minutes starting up hosts on the network to ensure all was well. The switch was configured properly - so it would be replaced with another switch while they played with the problem switch in the lab (someday... someday).This offsite analysis hits a key point in troubleshooting - the devastating failures are typically easier to spot. They scream at us. They stomp their feet and throw things. All they want is to have someone listen.

Enjoy life one bit at a time...
Laura

Join us at Summit 09 on December 7-9th! You'll get a copy of NetScanTools Pro and 3 full days of hands-on individual and group labs focused on troubleshooting and security. Download the Summit Information Guide. All Access Pass members receive a 50% discount to Chappell Summit 09. Don't miss it!

SNMP Snooping

Summit 09 Bonus: Licensed NetScanTools Pro - a $249 ValueAll Summit 09 attendees will receive a full licensed copy of NetScanTools Pro - a $249 value.

One of the labs for Summit 09 deals with SNMP snooping - locating information about a device by taking advantage of available MIB (Management Information Base) data through SNMP walking. Networks abound with SNMP-based devices - we can use the Port Scanner tool to generate a simple UDP scan for port 161 to discover those SNMP devices.

In NetScanTools, I discovered a few network printers supporting SNMP. I entered the IP address of one of the printers and selected the WALK action for the Object ID (OID) .iso.org.dod.internet. I left the community string at the default as I was certain no one had changed it since the printer was plugged in.

The result - a 24-page document filled with information about that printer and the other devices on the wired and wireless networks. The standard printer information was puked out as expected, but this SNMP snoop yielded loads more information:

• ARP table listing devices on the wired and wireless network
• MAC layer In/Out statistics (including errors)
• TCP In/Out statistics (including errors)
• UDP In/Out statistics (including errors)
• ICMP In/Out statistics (including errors)
• Routing table
• List of all received/transmitted ICMP packets
• SSIDs, channel numbers and signal strength of local WLANs - not just the WLAN that the printer was on and not just on the channel the printer was on

As I started playing a bit more and finding other unique SNMP devices, I realized I needed to load some new MIBs - a MIB is a database of objects. I found hundreds of MIBs available online at www.oidview.com/mibs/detail.html.

One of the coolest features in NetScanTools' SNMP tool is the ability to determine listening ports on the target without using a port scan. By generating udpLocalPort and tcpConnState queries, I could get the list of open UDP and TCP ports directly from the source. Using NetScanTools we can discover SNMP devices on the network, load an unlimited number of additional MIBs and perform a dictionary attack to identify the community string used by SNMP devices.

Join us at Summit 09 on December 7-9th! You'll get a copy of NetScanTools Pro and 3 full days of hands-on individual and group labs focused on troubleshooting and security. Don't miss it! Download the Summit Information Guide from www.chappellU.com. All Access Pass members receive a 50% discount to Chappell Summit 09.

Enjoy life one bit at a time!
Laura