Too many long nights and early mornings were partially to blame for the precipitation of a sense of panic that ensued when the following was seen in ISA's live query:
So, a quick search for the destination port of 5571 turns up: Trojan "Lamer Variant".ISA: Unidentified IP Protocol: Source Port 1175, Destination Port 5571
The next step was to figure out what the program/service was and where it was.
The inital PortQry using the GUI version turned up (unknown service) for the sending port of 1175. That was not too encouraging.
SysInternals' Process Explorer was also turning up nothing out of the ordinary.
The next step was to run the SysInternals Rootkit Revealer. After 45 minutes of scanning - this particular server has huge arrays - nothing out of the ordinary seemed to be there. The scan kept on going with nothing to show for it.
The Symantec A/V on the server was up to date and running with no indications of any interference.
By now, the panic has set in, and the thoughts swirling around were along the lines of, "Mr. Client, we need to perform a Swing Migration in the next 5 minutes." Not really a bad thing given they have a secondary AD server that also has the arrays mirrored. Or is it? With the possibility of rootkit infection, we may be pulling just the data from backups.
Mr. Client's reaction would probably not be too happy. :(
So, as a final effort before having to consider the above meeting, a full port analysis was needed - just in case.
In the command line PortQryV2 directory the following was done:
- portqry -local -l serverlog.txt [Enter]
After the serverlog.txt file was created, we opened it in NotePad, and did a find for 1175 to see if anything came up and low and behold:
Bingo. Open the Task Manager and shutdown the javaw.exe service, and the UDP errors in ISA disappeared.Process ID: 2476 (javaw.exe) on UDP Port 1175
Java is required for the Intel Management software that runs on the server. We had updated it during the last update run last week on that particular SBS box. So, something has changed in the program.
The DNSStuff.com report for the IP:
Reverse DNS for 229.111.112.12
Details:
strul.stupi.se. (an authoritative nameserver for 229.in-addr.arpa., which is in charge of the reverse DNS for 229.111.112.12)
says that there are no PTR records for 229.111.112.12.
A Whois for the mentioned .se server turned up Switzerland with no details. Not sure what Java is up to there.
For now, we will leave Java running, but not allow the UDP communication to pass through ISA to that 229 IP. We may even place a full ISA application restriction against it just in case.
After all of that, a huge sigh of relief and a little Irish kick! We got to smile and say, "Good day" to Mr. Client on our way out. ;)
UPDATE 2007-10-12: One thing that wasn't considered was searching for the actual IP listed. It seems that the IP is drawing people to the blog via search.
A question on Experts Exchange: Possible Hacker 229.111.112.12 mentions that the IP address may be an internal "Multicast" IP for the 229/8 range.
It would possibly explain why the Destination Network in ISA was Internal and the Source was the Local Host. It would also explain why there were no rDNS PTR records for the IP.
From the Internet Assigned Numbers Authority we have the following document: Internet Protocol V4 Address Space that indicates:
229/8 Sep 81 IANA - MulticastPersonally, this is not one of my strong points. So, please feel free to comment on whether this is the right direction or are we barking up the wrong tree?
Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
*All Mac on SBS posts are posted on our in-house iMac via the Safari Web browser.
It would have been wise to analyse the javaw.exe process using Process Explorer to see what Java app it was actually running.
ReplyDeleteAlso, TCPView would have given you the process fairly quickly.
Yes, I did not clue in on the fact that something else could have been riding on top of the Java process.
ReplyDeleteThe strange thing is that it has completely disappeared. There are no out of the ordinary TCP/UDP records going by in the ISA logs.
Watching the logs this morning, one of the client machines started firing off the same error code for the same external port to a different IP. It was there for a minute, but soon stopped.
So, it looks like we will need to do more digging.
Any other tool recommendations for rooting out the source of the problem?
Thanks for your input.
Philip
Philip,
ReplyDeleteThe first thing you should of noticed was 229.xxx.xxx.xxx is within the multicast address space.
The second item to note is the IANA analysis of reverse DNS lists the record type as MCAST-NET, signifying multicast. The Switzerland DNS record you saw was one of 3 or 4 listed in the recordbase and probably references IANA servers, insignificant.
The 3rd item found was the source local IP at 0.0.0.0 and the destination IP at *.*, which signifies a calling internal "inter process communication" (IPC) request, (0.0.0.0, since IPC's don't have IP addresses), to a destination multicast address/port of *.* (*.* because it's multicast (again) and the port query command cannot figure out which IP and port the process is calling.
One of the items you missed was the port query output referenced a PID which even task manager could have found, assuming the program was still running.
Javaw.exe was noted and it's a java runtime which a few programs do use for operation.
You can simply bring up the offending computer's installed software inventory list comparing it with the 2nd offending machine, while querying both computer operators as to what they "might" have remembered doing at that time.
Look for the installed program Azureus, which is a Java BitTorrent client.
Chris Knight was/is correct about Process Explorer.
Proper use of Process Explorer would note the offending program parent and the associated commandline (in properties) that started the process, only if it was still alive.
The best and still FREE program to use for tracking down LIVE processes is Process Explorer.
Since you know "which" computer it came from, another way I would catch the offending program is to run a packet sniffer (Free Wireshark) filtered to list multi-cast address/port from the offender. Later during the day, I'd check the log and read the packet contents, which may give a clue.
Or...
Doesn't ISA have an alert feature you could point towards the offending IP addresses? Once it fires off, call the user and growl demandingly "What in the hell do you think you are doing?!!"
If they are attempting file sharing, I almost guarantee that they will immediately stop. Hehe. :-)
Now that you know WHO is doing it, and approximately WHEN, it should be much easier to find the offending parent process.
Not sure if there is a program that can timestamp and list all executables run that day.
[SIZE=3][I][FONT=Microsoft Sans Serif][COLOR=DimGray]ZT3000
Beta tester of "0"s and "1's"[/COLOR][/FONT][/I][/SIZE]
Gee, I can't even get simple things like my signature line correct.
ReplyDeleteZT3000
Beta tester of "0"s and "1's
Phil,
ReplyDeleteI just noted that this computer is a server, so now you get to gruffly growl at an administrator. Whee...
Among other programs that use this, could it be java to sql related?
ZT3000
Beta tester of "0"s and "1's
ZT3000,
ReplyDeleteThe ISA logs can be analyzed for all traffic passing through both of the server NICs.
After spending a good chunk of time anaylzing the what/where/when of the inital error on the server as well as the one workstation it showed up on, it may actually tie into the DHCP and ISA error we received on the same server.
The error may actually be a legitimate process ... but caught in ISA due to a possible rule conflict as expressed by Amy in the comments section of the mentioned post.
It is the first time I have seen this one, but the IP mentioned is definitely bringing others here so it is not an "uncommon" occurance.
Thanks for adding your presence to the blog. It is truly appreciated.
Philip
Oh, and ZT300, I will do a quick scan of the system indicated to look for any BitTorrent clients.
ReplyDeleteOver the years, there have only been a very few number of users that we have had to deal with that were in some way troublesome in this regard.
Thanks again,
Philip
Are you still working on this project? I have many backed up and current resources from a corrupted iBook (Mac) from this domain. Also - .se is a domain in Sweden. I just found a post where I wrote to Just Answer. My grammar is pretty bad but I was typing quickly. You can do a search for strul.stupi.se and it came up for me in the top 4. I actually purchased a new Mac and the Trojan was transferred to it right out of the box via bluetooth the minute it was plugged in. Unbelievable.
ReplyDeleteLooking at a similar situation, these packets are coming from a Java runtime initiated from an executable called "VivaldiFramework.exe" running as a service (MRmonitor) which forms part of the LSI Logic MegaRAID Storage Manager.
ReplyDeleteThe machine (HP xw8600 workstation) has a LSI Logic MegaRAID 8888ELP SAS array controller installed.
The storage manager monitors the array controller (and attached discs) and provides the user interface to the configuration process.
Cheers, Colin.
Hi,
ReplyDeleteLike, Colin Butcher, same here:
multicast (229.111.112.12) by LSI MRmonitor (via VivaldiFramework and javaw.exe).
Reported by Wireshark as bad packets on the network.
Over the network other LSI RAID controllers can be administered. At startup of the UI you are presented with a list of controllers (IP addresses). I guess the latter are found via the above mentioned multi cast.
Never liked the LSI RAID software. Guess this another reason to do so.
As Colin Butcher said, LSI Logic MegaRAID Storage Manager uses such multicast traffic (229.111.112.12:3071) to manage LSI MegaRAID controllers.
ReplyDeletereadmeMSM
Regards,
openredes