From mikes at kuentos.guam.net Wed Feb 1 13:12:33 2012 From: mikes at kuentos.guam.net (Michael D. Setzer II) Date: Wed, 01 Feb 2012 22:12:33 +1000 Subject: [Udpcast] using udpcast with 1G ethernet. Message-ID: <4F292C31.27793.10F4F0@mikes.kuentos.guam.net> I just got a 24 port 1GB switch and my first attempt definitely didn't give the results I was hoping for. Perhaps there are some tuning options that might improve the options. First, I created an ntfsclone lzop compressed image of the Windows partition that is uploaded to one of the machines connected with the 1GB switch. That took about 11 minutes. I then made sure that all the machines had the nics running at 1G. Originally, two machines were only connecting at 100M, but replacing the connector on 1 and just recrimping the other got them all to connect at 1G. The cables are all Cat5e, but they are a few years old. I have a script that using udpcast to transfer this image file to all the other 19 machines in the lab connected to the machine. I was hoping to get a better speed when switching to the 100M to the 1G, but wasn't looking for a 10 times increase, since I knew that the hard disks only will do about 55MB/ second accordng to hdparm. But in my first test it took 31 minutes to transfer the file using udpcast?? Image file is about 12GB. Here is the end of the listing. bytes= 12 221 254K re-xmits=32506028 (378.1%) slice=0112 - 15 Timeout notAnswered=[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18] notReady=[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18] nrAns=0 nrRead=0 nrPart=19 avg=252 bytes= 12 229 698K re-xmits=32529469 (378.2%) slice=0112 - 3 Transfer complete. Didn't notice any machine giving errors, since it seemed to always list all 19 machines. The re-xmits seems to be an issue, but perhaps some other setting with the Linux (Fedora 16) or udpcast options. The script is run from the machine machine, and it links to the other 19 machines and then transfers. Partition /dev/sda6 is is a ext3 partition that is separate on all machines to store images, that can then be used to reimage individual machines with no network traffic. if [ -e /sda6/d7-xp.lzo ] ; then echo "SDA6 already mounted" else sudo mount /dev/sda6 /sda6 fi cmd="hostname && mount /dev/sda6 /sda6 && cd /sda6 && ./udp-receiver --no-progress --file d7-xp.lzo && cd / && umount /sda6 " #cmd="udp-receiver --file d7-xp.lzo " plink -t 192.168.7.201 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.202 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.203 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.204 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.205 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.206 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.207 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.208 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.209 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.210 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.211 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.212 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.213 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.214 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.215 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.216 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.217 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.218 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.220 -l root -pw xxxxxx "$cmd" & sleep 20 cd /sda6 time udp-sender --full-duplex --file d7-xp.lzo --min-receivers 19 --min-wait 9 I'm hoping there are some more experience with using udpcast in 1G environments. Currently running another test with sending to just two machines, but seems be showing similar results. +----------------------------------------------------------+ Michael D. Setzer II - Computer Science Instructor Guam Community College Computer Center mailto:mikes at kuentos.guam.net mailto:msetzerii at gmail.com http://www.guam.net/home/mikes Guam - Where America's Day Begins G4L Disk Imaging Project maintainer http://sourceforge.net/projects/g4l/ +----------------------------------------------------------+ http://setiathome.berkeley.edu (Original) Number of Seti Units Returned: 19,471 Processing time: 32 years, 290 days, 12 hours, 58 minutes (Total Hours: 287,489) BOINC at HOME CREDITS SETI 11801304.490587 | EINSTEIN 7206065.819851 ROSETTA 4194793.509365 | ABC 10747635.646984 From tyler at marieval.com Wed Feb 1 19:09:52 2012 From: tyler at marieval.com (Tyler Montbriand) Date: Wed, 1 Feb 2012 12:09:52 -0600 Subject: [Udpcast] using udpcast with 1G ethernet. In-Reply-To: <4F292C31.27793.10F4F0@mikes.kuentos.guam.net> References: <4F292C31.27793.10F4F0@mikes.kuentos.guam.net> Message-ID: <5B75CB67A3BB4418BE0D18A62141C062@Lab07> What "gigabit" switch did you get? I've had several lemons. Check ethtool -S eth0 if you can, if you see lots of RX_PAUSE, your switch is cheating, spamming pause packets to slow down the traffic to rates it can handle. It also shows receive errors. The client which didn't negotiate gigabit on the first try is strange. Having to force speed+duplex is one thing, but if the problem GOES AWAY, that suggests cable trouble to me. I haven't had negotiation problems that just go away when you try again. Negotiation doesn't test cable quality, it selects the maximum both support; it should've picked the maximum speed, first try, unless the cable was so bad negotiation just plain failed; after which both ends could've picked the same sane default. Also, if you have one slow receiver -- not necessarily a network problem, a slow disk could do it -- everything will wait for the slow node. -----Original Message----- From: udpcast-bounces at udpcast.linux.lu [mailto:udpcast-bounces at udpcast.linux.lu] On Behalf Of Michael D. Setzer II Sent: Wednesday, February 01, 2012 6:13 AM To: udpcast at udpcast.linux.lu Subject: [Udpcast] using udpcast with 1G ethernet. I just got a 24 port 1GB switch and my first attempt definitely didn't give the results I was hoping for. Perhaps there are some tuning options that might improve the options. First, I created an ntfsclone lzop compressed image of the Windows partition that is uploaded to one of the machines connected with the 1GB switch. That took about 11 minutes. I then made sure that all the machines had the nics running at 1G. Originally, two machines were only connecting at 100M, but replacing the connector on 1 and just recrimping the other got them all to connect at 1G. The cables are all Cat5e, but they are a few years old. I have a script that using udpcast to transfer this image file to all the other 19 machines in the lab connected to the machine. I was hoping to get a better speed when switching to the 100M to the 1G, but wasn't looking for a 10 times increase, since I knew that the hard disks only will do about 55MB/ second accordng to hdparm. But in my first test it took 31 minutes to transfer the file using udpcast?? Image file is about 12GB. From mikes at kuentos.guam.net Wed Feb 1 21:46:38 2012 From: mikes at kuentos.guam.net (Michael D. Setzer II) Date: Thu, 02 Feb 2012 06:46:38 +1000 Subject: [Udpcast] using udpcast with 1G ethernet. In-Reply-To: <5B75CB67A3BB4418BE0D18A62141C062@Lab07> References: <4F292C31.27793.10F4F0@mikes.kuentos.guam.net>, <5B75CB67A3BB4418BE0D18A62141C062@Lab07> Message-ID: <4F29A4AE.4342.1D52EC@mikes.kuentos.guam.net> On 1 Feb 2012 at 12:09, Tyler Montbriand wrote: From: "Tyler Montbriand" To: Date sent: Wed, 1 Feb 2012 12:09:52 -0600 Subject: Re: [Udpcast] using udpcast with 1G ethernet. > What "gigabit" switch did you get? I've had several lemons. Check > ethtool -S eth0 if you can, if you see lots of RX_PAUSE, your switch > is cheating, spamming pause packets to slow down the traffic to rates > it can handle. It also shows receive errors. > It is a Dell 2724 switch I got off ebay and just downloaded the manual, so am complete new to it. I did a test trying to use just two receivers to see if I could get better results, but was getting about the same results. Then did a single client test and got speeds of about 290MB with the two systems doing each separately, but when I tried to do them together afterwards it was giving like 2MB. I know that single transfers are unicast, so it seems to be something with the multicast setup. Have to look into getting in the switches management to setup the storm settings, but its default IP is not what this lab is on. Manual says 192.168.2.1 > The client which didn't negotiate gigabit on the first try is strange. > Having to force speed+duplex is one thing, but if the problem GOES > AWAY, that suggests cable trouble to me. I haven't had negotiation > problems that just go away when you try again. Negotiation doesn't > test cable quality, it selects the maximum both support; it should've > picked the maximum speed, first try, unless the cable was so bad > negotiation just plain failed; after which both ends could've picked > the same sane default. The cables were made by our MIS department long ago, and the one I replaced had the wires to far out, and the other one I just recrimped and it then connected. Since I got a 280MB speed on the two other machines I selected, it doesn't seem to be a cable issue with those 3 at least. > > Also, if you have one slow receiver -- not necessarily a network > problem, a slow disk could do it -- everything will wait for the slow > node. I have seen this in the past with machines, where one system would slow down the others, but it generally would just report that one machine not responding. In this case, the messages always seem to say that all the machines not responding, so it looks like it is the switch that is doing something. Even with just two machines it kept reporting [0,1]. Thanks. Might even try using the broadcast options, and see what that does. This lab is on its own 192.168.7.x network off a linux server that has 9 ethernet ports, so it should isolate from rest of campus, or just disconnect it for tests? > > -----Original Message----- > From: udpcast-bounces at udpcast.linux.lu > [mailto:udpcast-bounces at udpcast.linux.lu] On Behalf Of Michael D. > Setzer II Sent: Wednesday, February 01, 2012 6:13 AM To: > udpcast at udpcast.linux.lu Subject: [Udpcast] using udpcast with 1G > ethernet. > > I just got a 24 port 1GB switch and my first attempt definitely didn't > give the results I was hoping for. Perhaps there are some tuning > options that might improve the options. > > First, I created an ntfsclone lzop compressed image of the > Windows partition that is uploaded to one of the machines > connected with the 1GB switch. That took about 11 minutes. > > I then made sure that all the machines had the nics running at 1G. > Originally, two machines were only connecting at 100M, but replacing > the connector on 1 and just recrimping the other got them all to > connect at 1G. The cables are all Cat5e, but they are a few years old. > > I have a script that using udpcast to transfer this image file to all > the other 19 machines in the lab connected to the machine. I was > hoping to get a better speed when switching to the 100M to the 1G, but > wasn't looking for a 10 times increase, since I knew that the hard > disks only will do about 55MB/ second accordng to hdparm. But in my > first test it took 31 minutes to transfer the file using udpcast?? > > Image file is about 12GB. > > > > _______________________________________________ > Udpcast mailing list > Udpcast at udpcast.linux.lu > https://udpcast.linux.lu/cgi-bin/mailman/listinfo/udpcast +----------------------------------------------------------+ Michael D. Setzer II - Computer Science Instructor Guam Community College Computer Center mailto:mikes at kuentos.guam.net mailto:msetzerii at gmail.com http://www.guam.net/home/mikes Guam - Where America's Day Begins G4L Disk Imaging Project maintainer http://sourceforge.net/projects/g4l/ +----------------------------------------------------------+ http://setiathome.berkeley.edu (Original) Number of Seti Units Returned: 19,471 Processing time: 32 years, 290 days, 12 hours, 58 minutes (Total Hours: 287,489) BOINC at HOME CREDITS SETI 11801304.490587 | EINSTEIN 7206654.049851 ROSETTA 4195943.642083 | ABC 10760546.984548 From tyler at marieval.com Thu Feb 2 16:08:52 2012 From: tyler at marieval.com (Tyler Montbriand) Date: Thu, 2 Feb 2012 09:08:52 -0600 Subject: [Udpcast] Drivers for NIC and HDD In-Reply-To: References: Message-ID: <7755C38972254E2E9A41796AC401E43F@Lab07> You don't need drivers for a hard disk -- you need drivers for the hard disk CONTROLLER your drives are plugged into. As such, telling us the model of the drive is little help - I can tell they're SATA, but not what controller you're using. Have you tried the AHCI driver? That works for very many SATA controllers. For the realtek card, try the r8169 driver. _____ From: udpcast-bounces at udpcast.linux.lu [mailto:udpcast-bounces at udpcast.linux.lu] On Behalf Of Rahul Rao Sent: Saturday, January 28, 2012 6:57 AM To: udpcast at udpcast.linux.lu Subject: [Udpcast] Drivers for NIC and HDD I was just looking out for drivers for NIC and Hard Disk. Their model numbers are as follows, NIC: 1) Realtek RTL8168C(P)/8111C(P) Family PCI-E Gigabit Ethernet NIC(NDIS 6:20) Harddisk: 1) WDC WD2500 AAJS-60M0a0 ATA DEVICE 2) Hitachi HDP725016GLA380 ATA DEVICE Can you suggest me any other driver which can be compatible with this model of the hardware. -- Regards, Rahul Rao Msc-CA, SICSR _____ No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1913 / Virus Database: 2109/4779 - Release Date: 02/01/12 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikes at kuentos.guam.net Sat Feb 4 00:42:09 2012 From: mikes at kuentos.guam.net (Michael D. Setzer II) Date: Sat, 04 Feb 2012 09:42:09 +1000 Subject: [Udpcast] Gigabit switch question? Message-ID: <4F2C70D1.32024.107B36@mikes.kuentos.guam.net> I got a Dell powerconnect 2724 gig switch, and have been testing it, but have not gotten the results I had hoped for. Have been trying a number of things, and only about to get speeds similar to using a 100M switch?? When transfering with 1 sender and 1 receiver I can get speeds of between 300M and 400M with no retransmits. Set max bitrate to 400M to avoid timeout messages. Have done that with two different clients with same results. When sending to the same two clients at the same time (or sending to 19 clients in another test) I only can get a speed of about 70M to 78M at best. With no options, it drops to 1M to 2M. Setting the max bitrate seems to reduce the rexmits, but the setting of 80M. I've tried it with --fec 8x8/128 and without, and doesn't seem to make much difference. It might be the something in the switch, but I have gone in, and disable the storm protection settings that I had though might be an issue early in the process? The switch does have some limitations. Had to use an old 98 machine with old firefox to access its web interface, since it doesn't seem to work with newer browsers. Might just be this switch doesn't do well with multi-cast. +----------------------------------------------------------+ Michael D. Setzer II - Computer Science Instructor Guam Community College Computer Center mailto:mikes at kuentos.guam.net mailto:msetzerii at gmail.com http://www.guam.net/home/mikes Guam - Where America's Day Begins G4L Disk Imaging Project maintainer http://sourceforge.net/projects/g4l/ +----------------------------------------------------------+ http://setiathome.berkeley.edu (Original) Number of Seti Units Returned: 19,471 Processing time: 32 years, 290 days, 12 hours, 58 minutes (Total Hours: 287,489) BOINC at HOME CREDITS SETI 11810129.071646 | EINSTEIN 7213450.559851 ROSETTA 4201950.390797 | ABC 10802915.987447 From knightsamar at gmail.com Fri Feb 10 13:12:30 2012 From: knightsamar at gmail.com (Knight Samar) Date: Fri, 10 Feb 2012 17:42:30 +0530 Subject: [Udpcast] Giving a talk on udpcast Message-ID: Hi, I am giving a talk on UDPcast this year in GNUnify (http://gnunify.in) Any advice ? Regards, Samar -------------- next part -------------- An HTML attachment was scrubbed... URL: