Lejon Posted February 15, 2015 Share Posted February 15, 2015 Hi guys,I have links to about 150,000 images and want to download them, what's the best way to do it without crashing or eating memory?p.s. 1: my CPU is i5-4430 and RAM is 8 GB.p.s. 2: i have tried IDM with much less number than that and it was disappointing.Any recommendation? Link to comment Share on other sites More sharing options...
BALTAGY Posted February 15, 2015 Share Posted February 15, 2015 If they are direct links you can add them to a txt file and open the txt with IDM (Internet Download Manager)Every link by separated lineDam i didn't saw you used IDM, but for me IDM will do it just don't add all links to 1 txt Link to comment Share on other sites More sharing options...
flash48 Posted February 15, 2015 Share Posted February 15, 2015 Try Teleport Pro. The app is a web crawler and can copy an entire web site to your hard drive.http://www.tenmax.com/ Link to comment Share on other sites More sharing options...
Lejon Posted February 15, 2015 Author Share Posted February 15, 2015 @flash48 @BALTAGYI already have the links in a text.i tried IDM before and it took it about 10 minutes to load 6000 links, it crashed with 45,000 links and blew my registry because it's saving its downloading information in there. so IDM is out of the plan. Link to comment Share on other sites More sharing options...
user@nsaneforums Posted February 15, 2015 Share Posted February 15, 2015 try Wget for Windows Link to comment Share on other sites More sharing options...
Lejon Posted February 15, 2015 Author Share Posted February 15, 2015 I did a quick search and i found it's a CLI software, and unfortunately I'm not that expert to deal with it.P.S. the topic isn't about your favorite download manager, it's about the software which can handle a lot of links at once (150,000) smoothly. Link to comment Share on other sites More sharing options...
user@nsaneforums Posted February 15, 2015 Share Posted February 15, 2015 I did a quick search and i found it's a CLI software, and unfortunately I'm not that expert to deal with it.P.S. the topic isn't about your favorite download manager, it's about the software which can handle a lot of links at once (150,000) smoothly.Wget can handle whatever number of linksjust edit the txt by adding the Wget comand on every link/line & run the batwget http://download.com/filename.pngwget http://download.com/filename.pngwget http://download.com/filename.pngwget http://download.com/filename.pngwget http://download.com/filename.pngwget http://download.com/filename.pngwget http://download.com/filename.png.... Link to comment Share on other sites More sharing options...
xanax Posted February 15, 2015 Share Posted February 15, 2015 curl or wget as user@nsaneforums sayor even without third party tool with command like this:if not exist c:\wallpapers mkdir c:\wallpapersbitsadmin /transfer /download http://hdwallpaperd.com/wp-content/uploads/distant_planet_3d-HD_11.jpg c:\wallpapers\distant_planet_3d-HD_11.jpgbitsadmin /transfer /download http://hdwallpaperd.com/wp-content/uploads/Free-Download-3D-HQ-Wallpaper.jpg c:\wallpapers\Free-Download-3D-HQ-Wallpaper.jpgbitsadmin /transfer /download http://hdwallpaperd.com/wp-content/uploads/desktop-backgrounds-hd-nature-wallpaper-hd.jpg c:\wallpapers\desktop-backgrounds-hd-nature-wallpaper-hd.jpgbitsadmin /transfer /download http://hdwallpaperd.com/wp-content/uploads/full-hd-animals-wallpapers.jpg c:\wallpapers\full-hd-animals-wallpapers.jpgbitsadmin /transfer /download http://hdwallpaperd.com/wp-content/uploads/hd-nature-wallpaper-3.jpg c:\wallpapers\hd-nature-wallpaper-3.jpgbitsadmin /transfer /download http://hdwallpaperd.com/wp-content/uploads/Hd-Wallpaper-Nature17_1.jpg c:\wallpapers\Hd-Wallpaper-Nature17_1.jpgbitsadmin /transfer /download http://hdwallpaperd.com/wp-content/uploads/hd-wallpaper-25.jpg c:\wallpapers\hd-wallpaper-25.jpg Link to comment Share on other sites More sharing options...
clubhouse Posted February 15, 2015 Share Posted February 15, 2015 Bulk Image Downloader?http://www.ghacks.net/2013/12/02/bulk-image-downloader-4-65-review/ Link to comment Share on other sites More sharing options...
Lejon Posted February 16, 2015 Author Share Posted February 16, 2015 @user@nsaneforums @xanaxthanks guys, that's really a smooth way, just a couple of megabytes in ram :)but it's a little bit slow being downloading image by image. so I'm thinking to speed it up by splitting the links on multiple batchs. or is there a setting i can do in a single batch to download multiple images simultaneously?and another thing.. can i use a username & password or cookies when needed?@clubhousei used to use that program some years ago and it was a good one, can grab pictures and download huge numbers of images without a fuzz. but i don't recall it can import and download direct links from text? or can it? Link to comment Share on other sites More sharing options...
dcs18 Posted February 16, 2015 Share Posted February 16, 2015 When I need to download stuff (not limited to images) in volume, I just switch my IDM to Automatic Scheduler mode - my favorite number of files to download at a time is 3.IDM continues to download one file after the other, silently - without User intervention (there is no additional resource hogging, either.) Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.