I recently purchased iTunes match thinking it would be great way to store all my music in the cloud. Everything went alright during the initial match, about 3500 songs out of 4500 were matched leaving about a thousand to upload. I knew uploading these tracks would take some time, but I had a lot of difficulties with the process. Match would just upload 1 or 2 (or no) songs and then pause, leaving me to click the Start button and have it scan my entire library again, try and upload, and have it fail after another couple of songs. At this rate I knew I was never going to get my music in the cloud.
While uploading, I noticed something interesting. Many websites I tried to visit actually timed out – Match was choking my bandwidth hard. I set up a ping to google.com and it was failing with a could not resolve host. Perhaps this is a problem with my router configuration, but still I wondered – could Match be hogging so much bandwidth that it actually prevented iTunes from keeping the connection alive with the Store, ultimately causing Match to disconnect? Self sabotage sounds crazy, but to test the theory I installed netlimiter and limited iTunes to ~75% of my upload bandwidth. After days of failed attempts, I left Match to do it’s thing while I went off to work. When I came back home, to my surprise Match was still uploading (200 songs complete). The problem is solved.
So if you’re having trouble with iTunes Match uploading your songs, follow the steps below and you should get Match uploading without failure.
- Download NetLimiter for Windows from netlimiter.com. Don’t worry – you’ll get a 30 day free trial. However it is worth the $30 for other uses you’ll find for it.
- Find out your peak upload speed. You can find this out from speedtest.net (make sure nothing else is uploading at the same time).
- Open up iTunes and NetLimiter.
- Find iTunes in the NetLimiter list, tick the checkbox under the UL Limit column, and click on the speed to change it.
- Speedtest gives us a result in Mbps, so change the dropdown to Mbits, and enter roughly 75% of your peak upload (for example if your peak upload was 0.33, enter 0.248).
- Let Match do it’s thing, preferably when you’re not using the computer.
I hope this helps some people out with their iTunes Match and uploading. If you have any questions, please feel free to leave them in the comments section below.
EDIT: it looks like Entonnoir might be a good tool for mac users.
If this post has helped you out, feel free to consider throwing a small donation my way.
31 comments
Thank you for such a marvellous, easy, elegant solution! So far (proverbially knocking on wood) iTunes Match is running smoothly since I throttled my upload to 75% of my already heavily restricted ISP upload (.62Mbps on a 20Mbps fibre connection). NetLimiter is an old application that requires MacOS that is compatible with PowerPC apps, which I believe was last supported by OS 10.5 Leopard, perhaps 10.4. so it was useless on 10.9 Mavericks. Entonnoir looks like it will do the trick, if you know iTunes ports. The Apple Support Knowledge Base article “Well known TCP and UDP ports used by Apple software products here http://support.apple.com/kb/HT6175?viewlocale=en_US&locale=en_US is useful. Essentially it looks like the ports to throttle for iTunes
As previously mentioned NetLimiter won’t run on a non-PowerPC compatible Mac OS-X, I did it differently though according to this I found here – see the second answer that starts “For those on 10.7 or newer” here – https://apple.stackexchange.com/questions/7426/bandwidth-shaper-or-bandwidth-controller-app-for-mac-os-x/63349#63349?newreg=bab3be73026c4647bd68bf964dac78d9
So I set it to .62Mbps (625 kbps) across the board using Apple Network Line Conditioner system preference available to anyone on the Apple Developer’s site may be 80 and 443 used for iTunes Store (I don’t know if iTunes Match uses either servers), and 3689 for is for “iTunes Music Sharing” (which may be the port used for the traditional network sharing function in iTunes preferences, or also Home Sharing).
As I write, State 3 of uploading unmatched songs continues consistently without hang, and rather speedily for the some 201 song batch I’m doing. It was hanging at 11 or less. Now it’s on 35 and over several minutes.
I am using Activity Monitor/Network tab and searched iTunes only to monitor network traffic here – and packets sent are higher than received. Before I trotted, the ratio was the opposite.
Great that it’s working for you! NetLimiter is only available for Windows and as far as I know does not have an OSX version at all. For OSX users I did put a link to Entonnoir at the bottom, but I haven’t used it personally. The Network Line Conditioner tool looks really useful, I’ve also successfully used ipfw from the command line which is really easy to set up too: http://hints.macworld.com/article.php?story=20080119112509736
Do I need to registrer to be able to change the speed? I cannot change the speed box/field….
Is this for NetLimiter or Entonnoir?
My setup was constantly crashing at step 3, even with limiting the bandwith to 200kbps. I deleted from my local library every file that was over 100mb and step 3 started working. I removed the bandwith limit and it kept working, so in the end, it really was a file size problem for me.
But would you have any single music file that was over 100MB? Or do you mean that during the upload it will even scan movies and such?
I had a few songs over 100mb, hour-long compilations etc.
I just noticed that, when i show the sidebar (from the view tab), I am presented with three clouds. One for music, films, and tv programmes. During step 3 of the match process, the music cloud is empty but the two other clouds have moving diagonal lines (showing that its accessing the cloud or loading info). Could this be slowing my match process down as they seem to be in an eternal loading loop? I have loads of music videos and mini tv episodes that I added (not bought from iTunes Store).
I am certainly very slowly making progress, but everytime i reach something like “uploaded 53 of 786 songs” but i notice the numbers showing how many songs are in the cloud only go up by 1 or 2 each time.
Thanks again.
As far as I am aware, iTunes Match is only for music. It doesn’t match Music Videos and TV Shows that weren’t purchased from the iTunes Store. You might just have to keep trying until it finally gets all your songs into the cloud.
Slightly off topic but regarding the iTunes Match issue. I’m having the same problems. It took me months to upload the 4900 of my 5800 songs. Now into the last 900 songs my iTunes Match can never even seem to reach Step 3. Instead it gets stuck on the last 20 odd songs it’s matching in step 2. Could this be the same problem?
Also strangely, (I’m using a PC btw) but in the last month or so whenever it gets to the end of step 2 my PC inexplicably shuts down. Could this be a result of iTunes Match? I’m literally only running iTunes during this process so I fail to see how it alone could cause my PC to turn off.
Finally in regards to all of this DNS/CDN stuff, I live in China and am using my UK iTunes account to use iTunes Match. Could this be causing a problem?
Thanks for your time.
Best way to test is to leave your PC on without iTunes running. If it doesn’t shut down, it’s iTunes that causes the problem. There might be one particular track causing the issue, check if you’ve got any massive tracks left to upload (1 hour compilations etc), and try removing them to see if it works.
I doubt that there would be problems using iTunes match abroad. I have an AU account and use it all over the world and I haven’t had any further problems since I limited the upload speed.
I see. Well it actually turned out my PC was dusty and clogged up thus overheating all the time.
Now that it works I have found that although it seems to get stuck on a song in step 2 it will eventually move onto the next song after a seemingly random amount of time (could be 30 minutes to an hour).
One question for you though. Is the speed limiting process only for step 3 of itunes Matching? Or would it also speed up step 2?
For me, the speed limiting technique was only for “Step 3: Uploading artwork and remaining songs”. However, during step 2, it needs to upload data about your library to the iTunes servers. So this could potentially be affected by the same problem. I’d try limiting the speed to see if it helps at all, although it sounds like you’ve got a different problem to be honest.
I’ll have a try and hope for the best.
I did also realise that I had uploaded an additional 50 songs (or probably matched as it never got to stage 3) so maybe I can hopefully, slowly but surely complete this arduous process.
Thanks for your help Christian 🙂
Christian et al – thanks for posting. The comments led me to what I think might be the root problem and an alternative solution that doesn’t involve throttling.
I’m running a Mac Pro on 10.8.5 connected to cable broadband 20Mbps/5Mbps. Recently I decided to try iTunes Match. Generally the process would halt during step 2 or 3 with unknown error -9810.
I tried several times using Entonnoir with various settings. Eventually I tuned it to throttle ports 80 and 443 using 1024 down (default) and 512 up (~ 75% of speedtest result) KBytes/s. That worked slightly better but still failed often.
Then it hit me — iTunes Match is being handled by a CDN server too many hops away.
Content Delivery Networks (CDNs) are great for handling high bandwidth services. In principal, the requests get routed to a fast server located geographically near to you. Typically, using your ISPs DNS servers will accommodate this targeting of local CDN servers. It’s good for everyone — fast delivery, better load balance, better QoS. Apple uses Akamai’s CDN networks for iTunes and other services.
I use an alternative DNS server, OpenDNS, on my router to shield my family from certain classes of websites. For the most part it does what it is supposed to without introducing any side effects. However, one known problem is that it can poorly resolve CDN addresses.
Normally, the local ISP’s DNS server will resolve to a local CDN server and speedy data transfer ensues. But the OpenDNS servers are not local to me. By letting OpenDNS resolve addresses, I’m letting it resolve to CDN servers that may be very far away. And to make matters worse, it can be resolving a massive number of users (from everywhere) to that server far away from me. That’s a recipe for timeouts and errors rippling back to my system. The same thing is known to sometimes happen with Google’s Public DNS server (8.8.8.8) and similar non-ISP resolvers. Plenty of variability depending on where you are on the network and what’s going on.
In the past I’ve experienced dead slow downloads from Apple while plenty of other sites were lightning fast. Disabling OpenDNS restored fast Apple downloads. Now it occurred to me that iTunes Match uploads are suffering the same fate.
So, I installed Unbound (a local caching DNS resolver)(installed using Homebrew) and configured separate forward-zones for several potentially relevant CDN domains:
akadns.net
akamai.net
akamaiedge.net
akamaihd.net
akamaitech.net
apple.com
edgekey.net
icloud.com
mzstatic.com
I pointed those zones towards my ISP’s DNS servers. Then I created a catch-all forward-zone for everything else and pointed it towards OpenDNS.
Problem solved!! Match uploaded more than 12,600 items ( 25 GB) in a single session (16+ hours) without throttling and while using the computer normally for other tasks.
This approach would also explain why some people sidestep the problem by connecting to a different network — they’re likely accessing different DNS servers that resolve to closer CDN servers than when they are at home.
Worked for me, hope it helps some of you!
That’s a very interesting post and highly informative, thanks for taking the time to share this on my blog! I actually don’t have the old machine I originally uploaded from on so I’m not sure if this could have been my problem too, however there is a possibility I was using Google’s DNS at the time. I have no doubt this will help many people.
Worked for me too, but I had to lower mine to a little less than half my upload bandwidth. I’m on Time Warner Cable, with a pro account at 50 MB/s down and 6 MB/s up (56 x 5.78 MB/s actual). I had to upload nearly 7,000 songs from my PC.
The NetLimiter out setting that worked best for me was 2.5 MB/s for iTunes.
Thanks so much for the tip… been frustrated with this for weeks!
So glad I found this post and comments. Over 15,000 songs in my iTunes and over half would not load to match. Downloaded Entonnoir and attempting throttled uploads now. Wish me luck!
Thanks again!
Karson
YES! This finally works! After spending days reading and trying things – I think this gets at the heart of the problem. Used Entonnoir with ports 80 and 443 set to max upload at 500KB/s (as per the comments). Thank you so much.
Finally! I’ve tried numerous suggestions posted in the Apple Support Forums (delete & re-add songs, hard reset iTunes Match (a PITA on 3 iPads, 4 iPhones, and 3 Macs), selectively remove songs that caused step 3 to crash, disable IPv6, manual iTunes Match upload, disable wired Ethernet and use WiFi) but continued to experience the inability to get the last 644 of 12,000 songs uploaded. As soon as you described the network stalls, the light bulb went off in my head. iTunes is creating (or at least contributing to) a network problem.
I set up a ping to a host outside my home network and let iTunes Match run. When step 3 started uploading, I saw complete packet loss for 5 seconds, and then ICMP packets arriving out of order for a few more seconds until the Match process erred and restarted.
Mac OS X already has a firewall built-in (IPFW) so limiting bandwidth/traffic shaping is a piece of cake:
ipfw pipe 1 config bw 256KBytes/s
ipfw add 1 pipe 1 src-port 443
ipfw add 2 pipe 1 dst-port 443
I set up a ping to a host outside my home network and then kept pushing the bw limit down until network traffic was no longer being impeded. Now, as I type this, my songs are uploading. Perfectly.
This bug seems to affect a small number of highly irritated users. I’ve read reports that uploading from a different WiFi network often solves the problem. The iMac isn’t that portable, so I didn’t try that. In case it helps someone, my iTunes media resides on a 27″ iMac 2009 connected via wired ethernet to a Trendnet gigabit switch, which is connected via wired ethernet to the 100MB Ethernet port of an original Time Capsule, which is connected via wired ethernet to a Motorola Surfboard 6121 connected to Comcast. Outside of this iTunes issue, I have absolutely no issues with my home network. During the 12+ months I have had this issue I replaced the cable modem (encouraging Comcast to roll out IPv6 on the West coast), the gigabit switch (old dlink failed), and had the 2012 AEBS in where the Time Capsule is now.
Glad the post was able to help, and thanks for the detailed explanation of your findings!
Awesome!! Great solution Christian! Excellent command detail Matt!
I finally decided to get iTunes Match to get music to my new iPad but experienced the restart of the matching over and over issue. Thank You for your efforts and for documenting your solutions for all to learn from.
chris 🙂
Do we need to change the download bandwith also, or just the upload?
You should only need to limit the upload, as Match shouldn’t be downloading anything while it’s matching your library.
Entonnoir works perfectly! Port 80 (HTTP) and 443 (HTTPS), the default UL is 1024 kBytes/s. I limited my UL on both ports to 512 kBytes/s and after 6 months of fighting with iTunes Match (and not really using iTunes out of frustration), IT FINALLY WORKS. The remaining 100 songs should be uploaded in the next hour and, more importantly, my networking isn’t crashing every 5 minutes because of iTunes. So happy I found this article!
Thanks for the comment, glad you got it sorted! It’s definitely a weird issue and it doesn’t seem to be affecting the majority of people. What router are you using?
Worked like a charm! thank you all!
I’m experiencing the same thing — 4 songs left to upload that are ~120 MB. Should be within the iCloud limits, but they take forever and iTunes Match becomes a bandwidth hog. I’m using a Mac too so NetLimiter won’t work.
I just edited my post, have a try of this: http://www.macupdate.com/app/mac/28072/entonnoir
Great idea…I am having this exact issue and thought it was strange that iTunes caused network traffic to come to a halt. I’ll give this a try, although I have a mac and need to find a tool like NetLimiter.
I just edited my post, have a try of this: http://www.macupdate.com/app/mac/28072/entonnoir