Parents, 'Puters and Painkillers

"Hi hon! How are you? How are the kids? I can't print"...

Being a technologist these days is like being the family doctor in the olden days (ok, well, family doctors are still of value but mostly for prescription drugs for fun I think.)

You know what it's like - your second cousin once removed calls - you haven't seen her since that embarrassing Thanksgiving when they pulled you into singing "Muscrat Love" with them while your inebriated Aunt tried to play the piano ("I haven't played since I was a child" - no kidding?!). [That's another story.]. "Hey... are you still into computers?" Uh... no. I'm now working at a humane beef ranch as an ozone protection analyst. Sorry.

In this case, my father was calling for help with printing. Guiding him to view the print queue won't work - the print queue icon seems invisible to him and the Start button is out of the question ("The start button... you mean the power button? Ok. I clicked it, but my computer screen is blank now."). First things first. Do you see a light on in the front of the printer ("Yes, honey. My desk lamp is always on.")? It would be a long, slow and painful process (looking for the real family doctor for those fun meds now) to guide my father to eventually unplug and replug in the printer USB cable on his laptop ("no, Dad... the printer cable doesn't plug into the wall socket...get out from under the table before you hurt yourself.").

The printer sprung to life and began printing the 32 copies of the 70-page document he'd sent to it before calling me. Rather than try to guide him through the process of clearing the print queue I just told him that there wasn't anything he could do about it. "Just get out the recycling bin, Dad." (Making notes to give Dad reams of paper next birthday and go out to plant something green while acknowledging the guilt of prioritizing my sanity over the environment).

You must have a certain level of compassion and empathy to work in the field of technical support. I really don't know how people take calls from someone like my father every day and still maintain a life of sobriety and love towards mankind. I think the key must be... Hang on... gotta cut this blog short... my Dad's calling... ("Honey... I've just downloaded Wireshark and I have a couple questions...") Gulp.

Laura
Family... can't live with 'em... can't DoS 'em (legally)

Did That Tech Just Tell Me to Go Ping Myself?

"Ping 127.0.0.1 and let's look for packet loss."

"Let's reinstall the operating system."

"Oh my gawd - didn't you know ping is illegal?"

"Ping takes away the addresses of others on the network."

"Not all laptops support networking, so that might be the problem."

"Did you plug in the wireless cable yet?"

Oh yes... I keep track of the amazing comments I've heard from hotel network technicians and most recently Comcast. Many of you know the story of "Bob, the Comcast technician from hell" who ended up being a trainer for the other network technicans. I can only hope Bob is now flipping burgers somewhere.

When one of my network connections began feeling last week I pulled out my tools and began to work on identifying where the problem was. I grabbed my traffic with Wireshark and noted the high rate of packet loss. Since I know that packet loss most often occurs at an inter-network device, I began running the graphical ping in NetScanTools. I could see the rate of packet loss was around 40%. Next I began a series of traceroute operations to see where I was losing packets - and BOOM! There it was. One of the routers consistently dropped packets along the path. I even went through that target to other hosts.

All I needed to do was let the Comcast technician know which router was the problem... right?

When the Comcast technician asked me to "ping 127.0.0.1", I tried not to gag. How could this 'technician' not know the basics of TCP/IP? She pronounced traceroute as "trace-ert" with absolutely no awareness that her ignorance was spilling out over the phone.

What I experienced here is the result of skipping basic training - it's really not her fault. I blame Comcast. And guess what...? Right now we are seeing companies restrict training budgets for the folks running their networks. We're going to pay a big price in the future with unskilled and out-of-date IT professionals.

Is your company restricting training? What are you doing to keep up? Does your management know the end result of de-valuing training? Where will we be in a few months... years?

I hope the free Wireshark training courses are helping out. We are focusing on getting sponsors to open up more free online training. Let your favorite vendors know they can do something great for the industry by sponsoring a free training course.

Now off to finish the Wireshark 101 handouts for class on Tuesday! Gerald (the creator of Wireshark) will be online again to answer your questions. I hope to see you there! Register at http://www.chappellseminars.com today.

Laura

July 7th - Wireshark Jumpstart Free (Sponsored by NetOptics)

The July 7th Wireshark 101 Jumpstart is sponsored by NetOptics. I approached NetOptics because my lab is filled with NetOptics taps... the Teeny Tap, my 10/100 aggregating tap, my 10/100/1000 regenerating tap and more.

In this Wireshark 101 Jumpstart, I'll be demonstrating the following features of Wireshark:

• Tapping into traffic
• Choosing the interface
• Capture filtering
• Display filtering
• Capturing to file sets
• Capturing with a ring buffer
• Altering the time column
• Display filtering
• Using the Expert Info Composite
• Defining profiles
• Reassembling streams

We already have over 1,000 registrations and only 1,000 people will be allowed to access the live online seminar. We'll open up the 'waiting room' online approximately 20 minutes before the session to allow you to get a place in the course.

See you on Tuesday, July 7th.

Laura

Laughing at Twitter Traffic!

It's true... I was laughing out loud today... at packets!

This project came out of thin air almost... I was preparing for a podcast with the ChannelWeb group (you can listen to it at http://community.crn.com/docs/DOC-1082). I was on the phone line early with the moderator and interviewers and making small talk.

I mentioned that I'd tried to do some Tweeting that morning and there were problems. I explained how I used Wireshark to determine the problem had nothing to do with my system. There seemed to be a problem with the twitter.com website.

When the interview started, Ed Moltzen (a very impressive Tweeter and interviewer) led the discussion back to my early morning problems with twitter.com. As I talked about the problem, it suddenly occurred to me that people might like to know what Tweet traffic looks like. I told Ed that I'd do an analysis of a Tweet after the podcast.

I did... I immediately got working on a clean trace showing just the Tweet. That was no easy feat since my host spewed all sorts of background traffic for unrelated processes. I began identifying and whittling away traffic that was unrelated. Finally - I sent my sample Tweet and created my analysis report. But I wasn't done...

TweetDeck was ripe for an analysis... and here's when life got really fun. It turns out that when you upload your Twitter picture it is placed on an Amazon Web Server (AWS) under the original file name. Each user has a unique user ID and the image is placed in that directory under a directory called profile_images. The picture names were hysterical!

• WhatSheWants
• MeNoWife
• Spoon_too_big
  

You can read the entire report at www.chappellseminars.com/projects.html. I also released the MAC World Domination project details at that location.

Register for the newsletter over at www.chappellseminars.com/newsletter.html to keep up with the latest projects in my lab.

Now - off I go... the packets are calling!

Laura

iPhone: You're Sexy, but You Talk Too Much

Last week at Sharkfest I blabbered on a bit about the chatty nature of my iPhone (3G). I equated it to a yapping Chihuahua on the network. I'm still playing around a bit with numerous trace files and will have some to give away soon, but I wanted to explain how to capture your iPhone traffic and understand one of the packets that you'll see over and over and over and (you get it) again in your traffic.

I'm hanging out today on my Vista 64 system that I host the live seminars from. (No... I do not have a sexy MAC on my desk - but I do have two televisions within 10 feet of me to constantly feed me my much-needed background noise through the day.)

Before launching Wireshark or turning on my iPhone - here's what I did:

1. I hooked up a powered USB hub and populated it with three AirPcap adapters.
2. I opened the AirPcap control panel and configured each adapter to listen to a different channel - channels 1, 6 and 11.
3. I added my encryption keys in AirPcap.

Now I launched Wireshark and selected the AirPcap Multi-Channel Aggregator interface for my capture. Then I turned on my sweet, sexy-looking iPhone and...

OUCH! I watched my iPhone locate the WLAN APs, but it did not make an authentication/association until 60 seconds after I entered my passcode. Perhaps it wanted a bit more of a commitment from me? Or flowers? Or a new case?

During the startup sequence there were some unique DHCP and ARP happenings (we'll cover in a later blog) and a slew of mDNS packets. So, you ask... what the heck is mDNS and do I want 'em on my WLAN? mDNS stands for multicast DNS and is used to discover local devices as part of the zeroconfig project definition (Apple calls it Bonjour - they are so cool!). You don't need a DNS server to discover mDNS-capable devices. mDNS runs over UDP port 5353. Just use a udp.port==5353 filter or the dns display filter in Wireshark to see all mDNS and DNS traffic or build a filter for all ip.addr==224.0.0.251 traffic (the IPv4 mDNS multicast address) or ipv6.addr==FF02::FB, in the case of IPv6.

Want to try it out? On your iPhone, search in the AppStore for mDNS Watch. It's free so install it and watch it list all the mDNS-capable devices around you. In my lab it discovered my HP Officejet Pro L7700 printer and it showed me the three ports that were open on that printer - ports 513, 80 and 9100. Hmmm... this could be interesting, couldn't it?

For more information on mDNS, visit http://files.multicastdns.org/draft-cheshire-dnsext-multicastdns.txt.

Now... back to that hot, sexy and really verbose iPhone to work on the strange DHCP and ARP behavior (much of which is related to Bonjour).

Wireshark v1.2 Enhancements

In this week's newsletter I got carried away with details about the next version of Wireshark - it almost became a book. This blog details some of the enhancements in Wireshark v1.2.

One of the hot features that many will be thrilled about is auto-completion of display filters! HALLELUJAH! Bad typicsts rejoice (I meant to make that mistake...). Type in "i" and possible filters are shown in a drop-down list. Add a "p" and a period ("ip.") and all the possible variations of filters starting with "ip." show up. This is going to save us all a lot of time!

I already talked a bit about the GeoIP stuff in the Newletter and I'll be blogging/teaching about this a bit in the coming weeks.

There are a few changes that might sneak up on you - for example, in the Expert Info Composite area, "Window is Zero" and "Window Full" have moved to Warnings, but "Retransmissions" was not moved over - "Fast Retransmissions" are already in the Warnings area. It would be nice to have both types of retransmissions in the same window. We do now have the individual item count as well as the summary count in the tabs now, which is really nice.

There were some usability enhancements as well. For example, Wireshark v1.2 now remembers you column widths and opens up with the last configuration profile you used (watch out for this one if you're accustomed to always starting with the default profile and having to switch over).

As far as bug fixes go, the NetFlow dissector bug that could "run off with your dog, crash your truck, and write a country music song about the experience" has been fixed. No kidding - that is in the 1.2 rc1 release notes from Gerald.

Something that you may not take advantage of quite yet (but we'll cover in future newletters and online training over at chappellseminars.com is the new support for pcap-ng, the next-generation capture file format. These trace files typically end in the extension .ntar, but the recommended extension is .pcapng. This new trace file will enable us to add metadata to our trace files.

Again... the developers did a great job with this version - kudos to them all!

I'll be moving over to the new version of Wireshark for all the chappellseminars.com courses as soon as the "official" release is completed. Register for a course today!

Note: [25% Discount Code: bcbsab - use for the new Wireshark Command-Line Tools: From Editcap to Tshark - July 13, 2009 @ 10:00AM PDT/GMT-7

Survey: Chappell Seminars "Take the Reigns"

Twitter: LauraChappell

Facebook: Laura Chappell

You Can't Hide!

You may be familiar with the standard old traceroute that relies on ICMP echo request and echo reply packets to identify the path to a target and verify the target reachability. If so... how many times have you not reached the target because they filter ICMP echo replies?

An example of this would be when you try to traceroute to http://www.microsoft.com/. You'll see right after you hit the msn.net domain routers you are left in the dust. It really isn't that unusual to block ICMP echo requests at servers - no one should be pinging them anyway, right?

Using TCP Traceroute
Using NetScanTools Pro, I typically use TCP traceroutes. In the Traceroute tool, click the Setup button and choose TCP (WinPcap). You can define the starting hop, timeout in miliseconds, and retries at this point, but I go directly down to the TCP Trace Specific area.

Here's how the TCP Traceroute works - NetScanTools sends out a series of TCP SYN (handshake) packets to the target. It increments the Time-to-Live (TTL) value in the IP header (just as an ICMP traceroute does) to locate routers along the path who respond with ICMP Time to Live Exceeded in Transit messages. When the hop count is high enough to allow the TCP SYN to make it to the target, that target MUST respond - hey those are the rules of TCP. The target must respond with either a TCP SYN/ACK (indicating the target port is open) or a RST (reset, indicating the target port is closed). In this case, we don't really care if the target port is open or closed - we're just trying to get the roundtrip time using traceroute.

Firewalled/Blocked Targets
Now we know the specs for TCP say the target must respond... but what if it doesn't? What could have happened. Well... either your TCP SYN packet never made it there or the TCP SYN/ACK or RST never made it back. Make sure you run your TCP traceroute a few times to ensure sporadic packet loss isn't to blame. Most likely it is likely a firewall or some other blocking device that in your way. You couldn't find the roundtrip time, but you did find a protected host.

FYI - NetScanTools Pro 2-for-1 Price
As you may know, NetScanTools is on my 'must have' list of tools for IT professionals. The new version (updated today) is available at www.netscantools.com. There is also a 2-for-1 sale online through June 15, 2009.

Learn More
In the upcoming "Trace Back to a Suspect Host" course (June 4) I'll demonstrate each form of traceroute along with numerous other invasive/non-invasive techniques for testing connectivity, paths, identities and relationships of targets. Register online at www.chappellseminars.com/sem-traceback.html.

Laura