Nestled in my trusty Sponge Bob Slippers and surrounded by mounds of tissues boxes, cough syrup with Codene, a temperamental foot heater, NetFlix on-demand and business cards of every Chinese restaurant in the neighborhood... I settled in to study signatures of Nmap's fascinating OS detection process. Ahhh.... comfort packets.... First - if you haven't picked up Fyodor's Nmap Network Scanning book - put that on the top of your to-do list (wait... squeeze "Sign up for Summit 09" just above that). You'll want to snuggle up with pages 177 -178. You can get the book at Amazon or try to reach Fyodor over at insecure.org - buy it directly and ask him to sign your copy - this is a hot book!
Here's the scoop - capture your traffic as you run nmap -sV -O -v against your target (version scanning, OS fingerprinting and verbose mode). Got permission, right? Good. Read on.
Nmap's OS fingerprinting process contains numerous unique packets - by building a series of butt-ugly color filters you can spot these strange packets easily with relatively low concerns of false positives (if you happen to find these packets being sent by another application you should still be concerned - it's weird behavior).
In looking through the trace file and referencing Nmap Network Scanning, I came up two color filters (both with butt-ugly background colors) that caught the majority of the unique packets generated during the scan.
Filter #1
(tcp.flags == 0x00) || (tcp.options.wscale_val == 10) || (tcp.options.mss_val < flags ="="" urgent_pointer ="="" flags="="0x02" flags="="0x02"> 0)
So shall we break this down a bit?
(tcp.flags == 0x00)
This looks for the null scans - TCP scans that have no TCP flags set.
(tcp.options.wscale_val == 10)
The TCP window scale value equal to 10. Although other TCP handshakes may use this value during the handshake process, it is unusual and listed in the book as one of the scan techniques and verified in the trace file of the OS fingerprinting process.
(tcp.options.mss_val <>
This one is a bit sticky - we're looking in the options section of the TCP header for a maximum segment size value smaller than 1,460. This did cause numerous false positives when I ran it on other trace files. Regardless, I like having this in my color filter because it points out some weird starting MSS starting value. As an option, I considered moving this to another color filter with a slightly lighter background color.
(tcp.flags == 0x29) && tcp.urgent_pointer == 0
This filter looks for the FIN, PSH and URG bits set in packets with the Urgent Pointer field set to 0.
(tcp.flags==0x02 && !frame[42:4] == 00:00:00:00)
Yeah - this is a strange one and brings up a change I'd like to see in Wireshark. This looks for packets with the SYN bit set only and the Acknowledgment Number field set at a non-zero value. So... what's the "frame[42:4]" all about? Well... Wireshark does not recognize the Acknowledgment Number field in the first packet of the handshake process as it doesn't have any use in that packet. I'd still like to see the field in those packets so I can filter on it though. I tried messing around with !tcp.ack==0 but that didn't work.
(tcp.flags==0x02 && tcp.window_size <> 0)
This looks for SYN packets with a small window size value and the window scale factor set to 0. This did hit some false positives in other trace files, but they were all hosts with strangely small window size values anyway - a bit of a concern to me anyway. I considered setting this as a separate color filter and may alter my color filters as I test this against more trace files.
Filter #2
I set another color filter for tcp.window_size < syn="="1 - it was a bit lighter in background. This color filter had lots of false positives in other trace files, but pointed out numerous TCP connections that were using non-optimal starting wndow size values.
Ohhh... look at all the pretty, er I mean ugly colors! This particular Nmap scan sequence screams "Halloween All Year Long!"
Join us at Summit 09 as we investigate other malicious traffic patterns! Register over at www.chappellseminars.com/summit09 and I'll see you December 7th!
Enjoy life one bit at a time!
Laura
