Strange SYN Scans and SpongeBob Slippers

Nestled in my trusty Sponge Bob Slippers and surrounded by mounds of tissues boxes, cough syrup with Codene, a temperamental foot heater, NetFlix on-demand and business cards of every Chinese restaurant in the neighborhood... I settled in to study signatures of Nmap's fascinating OS detection process. Ahhh.... comfort packets....

First - if you haven't picked up Fyodor's Nmap Network Scanning book - put that on the top of your to-do list (wait... squeeze "Sign up for Summit 09" just above that). You'll want to snuggle up with pages 177 -178. You can get the book at Amazon or try to reach Fyodor over at insecure.org - buy it directly and ask him to sign your copy - this is a hot book!

Here's the scoop - capture your traffic as you run nmap -sV -O -v against your target (version scanning, OS fingerprinting and verbose mode). Got permission, right? Good. Read on.

Nmap's OS fingerprinting process contains numerous unique packets - by building a series of butt-ugly color filters you can spot these strange packets easily with relatively low concerns of false positives (if you happen to find these packets being sent by another application you should still be concerned - it's weird behavior).

In looking through the trace file and referencing Nmap Network Scanning, I came up two color filters (both with butt-ugly background colors) that caught the majority of the unique packets generated during the scan.

Filter #1
(tcp.flags == 0x00) || (tcp.options.wscale_val == 10) || (tcp.options.mss_val < flags ="="" urgent_pointer ="="" flags="="0x02" flags="="0x02"> 0)

So shall we break this down a bit?

(tcp.flags == 0x00)
This looks for the null scans - TCP scans that have no TCP flags set.

(tcp.options.wscale_val == 10)
The TCP window scale value equal to 10. Although other TCP handshakes may use this value during the handshake process, it is unusual and listed in the book as one of the scan techniques and verified in the trace file of the OS fingerprinting process.

(tcp.options.mss_val <>
This one is a bit sticky - we're looking in the options section of the TCP header for a maximum segment size value smaller than 1,460. This did cause numerous false positives when I ran it on other trace files. Regardless, I like having this in my color filter because it points out some weird starting MSS starting value. As an option, I considered moving this to another color filter with a slightly lighter background color.

(tcp.flags == 0x29) && tcp.urgent_pointer == 0
This filter looks for the FIN, PSH and URG bits set in packets with the Urgent Pointer field set to 0.

(tcp.flags==0x02 && !frame[42:4] == 00:00:00:00)
Yeah - this is a strange one and brings up a change I'd like to see in Wireshark. This looks for packets with the SYN bit set only and the Acknowledgment Number field set at a non-zero value. So... what's the "frame[42:4]" all about? Well... Wireshark does not recognize the Acknowledgment Number field in the first packet of the handshake process as it doesn't have any use in that packet. I'd still like to see the field in those packets so I can filter on it though. I tried messing around with !tcp.ack==0 but that didn't work.

(tcp.flags==0x02 && tcp.window_size <> 0)
This looks for SYN packets with a small window size value and the window scale factor set to 0. This did hit some false positives in other trace files, but they were all hosts with strangely small window size values anyway - a bit of a concern to me anyway. I considered setting this as a separate color filter and may alter my color filters as I test this against more trace files.

Filter #2
I set another color filter for tcp.window_size < syn="="1 - it was a bit lighter in background. This color filter had lots of false positives in other trace files, but pointed out numerous TCP connections that were using non-optimal starting wndow size values.

Ohhh... look at all the pretty, er I mean ugly colors! This particular Nmap scan sequence screams "Halloween All Year Long!"

Join us at Summit 09 as we investigate other malicious traffic patterns! Register over at www.chappellseminars.com/summit09 and I'll see you December 7th!

Enjoy life one bit at a time!
Laura

SSL/TLS Flawed: Using Wireshark to Decrypt Attack Traces from PhoneFactor

It seemed such a coincidence, I sent out a teaser for a project underway and alluded to the security implications - the project, however, was not related to the SSL/TLS vulnerability that hit the public last Thursday.

How bad is this SSL/TLS vulnerability? Amazingly horrid! Listen Up! (MP3 - 1MB) Click here to download Ron Nutter's interview with Steve Dispensa (or grab the .zip file here) - one of the PhoneFactor guys who demonstrated the vulnerability to a working group of affected vendors and representatives of various standards committees.

Read Up!
Steve Dispensa and Marsh Ray of PhoneFactor wrote an 8-page overview of the issue which is based on the TLS renegotiation process. The figure below shows the basic SSL/TLS handshake process.

In Wireshark, the display filter ssl.record.content_type == 22 extracts SSL/TLS handshake packets.

The document written by Steve and Ray defines the security issues demonstrated against recent Microsoft IIS and Apache httpd versions. In essence, the renegotiate attack method defined is used to inject malicious code into the "secure" connection.

One of the most interesting areas of the document focuses on the use of request splicing in which two HTTP requests are combined. The first request triggers the renegotiation while the second request effectively comments out the first request and overrides it with the malicious one.

Analyze the Attacks Yourself
Download the PhoneFactor document, numerous trace files (including decryption keys), protocol diagrams and details here.

Hint: In Wireshark, disable the Preferences > TCP > Allow Subdissector to Reassemble TCP Streams to view the SSL/TLS handshake more clearly.

Step 1: Get the Traces/Keys
Download and extract the files into a directory called "ugly". (Again - download from here.)

Step 2: Set up SSL with Keys
Private keys to decrypt the traces are in the 'caps' and 'certs' directories. For simplicity sake, I recommend you create a \keys directory and copy all the keys there.To decrypt the client_init_renego.pcap file, I used Preferences > Protocols > SSL and entered the following value:

192.168.80.125,443,http,c:\users\laura\keys\ws01.mogul.test.key

When you have successfully set up decryption, your traffic should indicate HTTP in the protocol column and, if colorization is enabled, the lovely lime-green color of HTTP traffic.

Step 3: Follow the SSL Stream
Once you have applied the decryption, you can right-click on one of the HTTP packets and select Follow SSL Stream to reassemble the traffic as shown below.

In the figure at left we can see the request to GET /evil.html and the x-ignore line for GET /index.html. This process of using the ignore header prefix is described on page 3 of the Renegotiating TLS.pdf document.

Inside the SSL/TLS Handshake - Another "Must Read"
Jeff Moser penned an impressive blog entry entitled "The First Few Milliseconds of an HTTP Connection" which analyzes the handshake process, selection of a cipher suite and use of the RSA algorithm. Read Jeff's blog here.

What's the Solution?

The document written by Marsh Ray and Steve Dispensa paints a pretty gloomy picture of possible remedies.

"There appear to be few silver bullets to address these issues."

Ultimately, the fix will require protocol changes - a laboriously painful process that can have unforeseen consequences related to compatibility problems. The paper forthrightly defines the possibility of 'breaking' as well as backwards-compatible protocol changes. It takes serious 01's to throw that 'breaking' term in there. It's no fun being the bearer of such bad news. What a hassle.

In the meantime, I imagine the efforts to exploit vulnerable SSL/TLS connections is underway - those malicious teams might be working longer hours than the vendor/committee teams focused on a resolution.

Big money is at stake.

Enjoy life one bit at a time!
Laura

Double-sided and Double-dumb Printing

Summit 09 Bonus: All Summit 09 attendees will receive a full licensed copy of NetScanTools Pro - a $249 value.

It's not easy going "green"... I mean - all we wanted to do was print the customized course manuals double-sided. What's the big problem?

WIRESHARK and I NEARLY BARFED!



First let me say I was printing to an HP OfficeJet Pro L7780 All-in-One printer - it supports auto-duplexing. The feature is implemented in the most funky way however! Watching the traffic live while the printer did it's 'dance of duplex' - I was shocked to see the lame process of auto-duplex printing.

I had to wonder if the HP printer group might be interested insome classes on networking.*

Ok... here's how this printer does duplex printing. It's automatic, so you don't have to do the old 'refeed the paper and hope it's in the right order' process. You just select auto-duplex and away it goes. The printer prints the first page and - while still holding the paper by the bottom edges, the console reads "Ink drying - please wait." After a while the paper is then sucked back into the printer (a moment that always makes my heart jump - I've heard the shredding sound of mangled paper too many times). The second side of the paper prints and we all sigh relief - another 2-sided page done!

The trace file showed that the printer creates the pause process by sending a Zero Window packet to the client - in essence "talk to the hand" (isn't that HP's little helper logo?). Each 'drying process' essentially was caused by halted flow of print data from my client to the printer. My client sent Zero Window Probe packets to ask 'What is wrong with you, buddy?" and the printer kept the traffic at bay by sending Zero Window Probe ACKs for about 30 seconds.

The printer's TCP window size did decrease down to an unacceptable zero so it wasn't lying about not having any receive buffers. It seems the printer doesn't clear out the buffer space consistently during the print process - it's buffer becomes depleted right at the point when the 'ink needs to dry" - perhaps it knows we'll all need to take a breath then so allowing the buffer to fill doesn't seem like a big deal - it's going to have to sit around idle anyway (anyone think about adding a nice 'ink fan' attachment to make my printing go faster?).

I decided to add a tcp.window_size column and filter on traffic coming from the printer. Using File > Export > File and choosing the displayed packets, I created a csv file with a column just depicting the TCP receive window sizes advertised by the printer. In Excel I selected only the tcp.win_size column and inserted a line graph.

BEAUTI-FRIGGIN-FUL!

Look at the intentional pattern of the window size values advertised with definite zero window conditions during the ink drying time. There's just got to be a better way!

What? Is my printer totally dumb? I didn't think so, but this is certainly dumb behavior. The file is only 13,296 KB! The printer has 64 MB memory standard. Why didn't the printer allow me to send the entire document to the printer, then buffer the data while the ink dried? Why did I have to sit around and wait for the ink to dry before being able to send the next page of data? The Wireshark summary of the data sent to the printer showed the transmission rate averaged around 0.431 Mbit/second - snooze...

I also look at the spooled file in my Windows\System32\spool\PRINTERS directory - holy schmoley! The spooled file (.spl) was 670,016 KB! Whazzup with that? The idea of researching the MS spooling process and .spl file format made my stomach turn again - but it did look like this Powerpoint Notes document was processed as a graphics file.

Then came the really interesting part - I printed twenty-five copies of the custom student manual. You'd think the printer would have buffered the first copy in memory and just pulled from that when it needed to make each successive copy - right? Nope. My client sent the entire file to the printer a second time, a third time, a fourth time... etc. Whoa... that's one dumb process! No turning off my system until the entire job was sent 25 times from the spool\PRINTERS directory. Barf.

Basic network analysis should be a mandatory requirement for any vendor making network-capable devices/applications - or else they might make a network-incapable devices/applications - oh wait - they do!



Enjoy life one bit at a time!
Laura

*Although this is the lamest way to get the job done, I swear by my HP printers - they take a beating and keep on spitting out surreal amounts of pages every month - I am a tactile, visual person - there's nothing like that printed version of the spec to snuggle up with at night! So much for going green. (The only green that will fly around here will be some big $ for printers that can buffer the entire doc once - without the 'talk to the hand process' - and then print extra copies from that!)

Tracing the Route

Summit 09 Bonus: All Summit 09 attendees will receive a full licensed copy of NetScanTools Pro - a $249 value.

During troubleshooting processes, a standard ping test is often used to check connectivity to a host and determine the round trip latency time. This process uses an ICMP Type 8 Echo Request and relies on an ICMP Type 0 Echo Reply.

Sometimes, however, the target won't respond to ICMP Echo Replies - either because it is configured to ignore these ICMP Echo Requests or because a device along the path filters these packets out so they don't reach the target.

I prefer to perform traceroute using NetScanTools' TCP option. Besides setting the TCP port and the sequence number settings, you can also set the MTU (Maximum Transmission Unit) to test the maximum packet size along a path.Another option available is to set the TCP window size - in our example I have set the window size field value to 16,384. In addition, you can define the payload - using a binary or text file.

Why would you use a big fat file for the test? Ahhh... my padawan - to test the MTU allowed through the path and consider putting a signature in the payload that should trigger an IDS or be logged by a firewall - multiple birds with one stone - connectivity testing, latency testing, IDS/firewall testing! Nice!

In the figure below, you can see my host 192.168.0.113 sending a series of TCP SYN packets - the target port is 79 (finger). The packets colored with a red background have an IP header Time-to-Live value less than 5 - a sure sign of a traceroute operation.

When we reach the target, a RST is generated in response. That's what gives us our round trip latency time.I appreciate why companies restrict ICMP-based traffic on their networks - and when I'm doing connectivity tests and latency tests, customizing my TCP-based traceroutes always sits on the top of my to-do list.

Enjoy life one bit at a time!
Laura

Storms Rip the Net

Summit 09 Bonus: Licensed NetScanTools Pro - All Summit 09 attendees will receive a full licensed copy of NetScanTools Pro – a $249 value.

I recognized the tone in the voice that day - the panicked sound of someone dealing with a non-functional network. In this case, the network was a critical one (I can't disclose the specific type of network or customer).

At approximately 3:34am, their critical network came crashing down - no connectivity for any hosts on the network. They'd placed Wireshark on the network and it too crashed within moments.

Ok... so there was something definitely cruising along the network wreaking havoc. I had to see those packets!With over 2,000 miles separating us, it would be a 'walk through the capture' process.

The first step - dump the GUI!

Wireshark comes with Tshark for command-line capture. The syntax used was:

tshark -c 100 -w gen1.pcap

The -c parameter indicates the number of packets to capture. The -w parameter is used to define the name of the trace file to create. Why only 100 packets? What? Well... if there is a catastrophic issue on the network that could kill systems that connect to it that quickly, it shouldn't take many packets to characterize that traffic.

Immediately upon capturing these 100 packets, I instructed the customer to disconnect from the network. You don't need network access to analyze captured traffic - trace files are processed through the Wiretap Library - directly off the disk.

The 100 packets told the story - an insane looping packet storming through the network at a blazingly rapid packets per second rate. When facing a traffic issue like this it is important to look at the IP header's Identification value. You need to differentiate between a looping packet or a series of individual packets sent from a 'killer host' (and I mean killer as in "network killer").

If the IP Identification field value is the same for all the packets, then the packet is looping somehow. If the packet has a different IP Identification field then the packets have each passed through the IP protocol separately from a host. It's an amazingly simple differentiation - and an important one.If the packets had unique IP Identification field values, we'd be looking at a single host causing the problems. We'd be delving into the MAC header of the packet to identify the location of the lousy host. (Having a master list of MAC addresses for all hosts on the network is imperative in that situation. Mark that down as something to do this week!)

In this case, all the IP Identification field contained the same value - this was a looping packet. We had an infrastructure issue. On this heavily switched network it seemed spanning tree was not doing its job. Poor spanning tree - no one really pays attention to it until it has a problem.

Being remote to the customer location, I could not look over their shoulder as they walked the network and shut down one switch at a time. It was in their hands now. I sat waiting for their call - waiting to hear if they'd found the culprit. I didn't wait long.

I waited 30 looooong minutes for the call even though hit had taken the client less than 5 minutes to find a switch that was looping traffic back through the network. They spent the other 25 minutes starting up hosts on the network to ensure all was well. The switch was configured properly - so it would be replaced with another switch while they played with the problem switch in the lab (someday... someday).This offsite analysis hits a key point in troubleshooting - the devastating failures are typically easier to spot. They scream at us. They stomp their feet and throw things. All they want is to have someone listen.

Enjoy life one bit at a time...
Laura

Join us at Summit 09 on December 7-9th! You'll get a copy of NetScanTools Pro and 3 full days of hands-on individual and group labs focused on troubleshooting and security. Download the Summit Information Guide. All Access Pass members receive a 50% discount to Chappell Summit 09. Don't miss it!

SNMP Snooping

Summit 09 Bonus: Licensed NetScanTools Pro - a $249 ValueAll Summit 09 attendees will receive a full licensed copy of NetScanTools Pro - a $249 value.

One of the labs for Summit 09 deals with SNMP snooping - locating information about a device by taking advantage of available MIB (Management Information Base) data through SNMP walking. Networks abound with SNMP-based devices - we can use the Port Scanner tool to generate a simple UDP scan for port 161 to discover those SNMP devices.

In NetScanTools, I discovered a few network printers supporting SNMP. I entered the IP address of one of the printers and selected the WALK action for the Object ID (OID) .iso.org.dod.internet. I left the community string at the default as I was certain no one had changed it since the printer was plugged in.

The result - a 24-page document filled with information about that printer and the other devices on the wired and wireless networks. The standard printer information was puked out as expected, but this SNMP snoop yielded loads more information:

• ARP table listing devices on the wired and wireless network
• MAC layer In/Out statistics (including errors)
• TCP In/Out statistics (including errors)
• UDP In/Out statistics (including errors)
• ICMP In/Out statistics (including errors)
• Routing table
• List of all received/transmitted ICMP packets
• SSIDs, channel numbers and signal strength of local WLANs - not just the WLAN that the printer was on and not just on the channel the printer was on

As I started playing a bit more and finding other unique SNMP devices, I realized I needed to load some new MIBs - a MIB is a database of objects. I found hundreds of MIBs available online at www.oidview.com/mibs/detail.html.

One of the coolest features in NetScanTools' SNMP tool is the ability to determine listening ports on the target without using a port scan. By generating udpLocalPort and tcpConnState queries, I could get the list of open UDP and TCP ports directly from the source. Using NetScanTools we can discover SNMP devices on the network, load an unlimited number of additional MIBs and perform a dictionary attack to identify the community string used by SNMP devices.

Join us at Summit 09 on December 7-9th! You'll get a copy of NetScanTools Pro and 3 full days of hands-on individual and group labs focused on troubleshooting and security. Don't miss it! Download the Summit Information Guide from www.chappellU.com. All Access Pass members receive a 50% discount to Chappell Summit 09.

Enjoy life one bit at a time!
Laura

The Game of VoIP

I always get that tingly excited feeling when I open up new trace files.

It's not unlike the feeling I have as I pick up the pile of cards dealt to me in a game of gin. What will the cards hold? What will I see in them? That is what analysis is to me - a card game. Last week I began my slide preparation for an upcoming set of projects related to VoIP analysis. I find VoIP traffic fascinating - its duality - signaling separate from voice - its trusting nature - using UDP - its frailty.

This VoIP project came up first in preparation for the VoIP labs for the Summit 09 training event. I wanted to include a lecture portion dealing with SIP and RTP behavior and point out the cool new organization of telephony-related options in Wireshark. I wanted to build some hands-on labs to allow students to identify the likely points on a network that they'd look to isolate problems with the calls. I'm energized with the process of preparing new lectures, new trace files and new hands-on exercises for Summit 09.But Summit 09 had to go on the back burner for a bit... another hand had been dealt - by Microsoft.

The Microsoft Webinars (Oct 6th and Oct 28th)
I have presented two webinars for Microsoft to date - one wildly popular (where we pulled in Microsoft's largest registration and attendance counts) and the other nearly empty (seems the old Microsoft marketing engine had stalled on that one and we didn't push it because it was open to partners only). If you want to join me at the Summit 09 event on December 7-9th, check out the Summit 09 Information Document at www.chappellseminars.com/summit09.html.

No RTP? A Lousy Deal?!
When I began systematically opening the VoIP files in my folder, I hit the first stumbling block of VoIP analysis. One of the first trace files I opened up did not contain the SIP packets for a call setup so Wireshark didn't apply an RTP dissector to the voice traffic - it just dumped me off at UDP. Hmmm... seems I have a hand with no runs or matches.

I've talked about the process of "Decode As" in the past for interpreting traffic traveling over non-standard ports. The same steps are quite common to use with VoIP traffic. I right clicked on a packet in the trace file and selected Decode As. On the right side, I chose one of the transport port numbers and selected RTP in the protocol list.

Voila! It's decoded as RTP! Another option is to enable the RTP Preferences "Try to decode RTP outside of conversations". The image at left shows the traffic of an RTP stream that Wireshark decodes as just UDP. Wireshark didn't see the SIP call setup process and it decodes the voice traffic as just UDP.



Viewing the RTP Streams
When Wireshark reached version 1.2, the menu bar was revised to include a Telephony item. Once you force the RTP decode (if required), you can select Telephony > RTP > Show All Streams.

Wireshark detected two RTP streams in my trace file once I forced the RTP decode on the traffic.

As you can see from the figure at left, Wireshark can now show me the RTP streams in the trace file - one stream going in each direction. We can also see that one of the streams has a 2.1% packet loss. Packet loss is ugly in VoIP communications - when a call experiences serious packet loss, the conversation quality suffers.There's an example of a G.711 audio file with 10% packet loss at VoIPTroubleshooter.com.

Playing the Hand That Was Dealt
Unlike gin or poker, we have to play the hand we were dealt in network analysis. We can only make the hand stronger by filtering out traffic that is not relevant and interpreting the possible causes for the packets that remain.

In the area of VoIP analysis, my favorite filter is sip || rtp and my favorite feature is the RTP playback feature. I like to set the time column to Seconds Since Previous Displayed Packet and export the trace file to csv format.

In Excel I sort on the source column first, the number column second and then graph out the time column to show me the variance between packets (the jitter value). I could use Wireshark's graphing capabilities to give me some of this information, but this is one of the areas where a formal spreadsheet program really shines. I exported the trace file to a csv format to graph out a portion of the jitter values.

VoIP at Summit 09
I'm anxious to get back to the Summit 09 preparations - to document all the tips and tricks of VoIP analysis. I've put together a few lab exercises already, but I'm keeping my eyes open for new hands to play - if you have any VoIP traces that I could use in Summit 09, please send them to me (laura@chappellU.com). Download the Summit Information Guide. All Access Pass members receive a 50% discount to Chappell Summit 09.

Enjoy life one bit at a time.
Laura