Creating Bootable Memory Sticks

My special interest is computers. Let's talk geek here.
Post Reply
User avatar
yogi
Posts: 9978
Joined: 14 Feb 2015, 21:49

Creating Bootable Memory Sticks

Post by yogi »

Things at the computer ranch are calm these days, but it hasn't been for the previous week or two. Some major disasters have occurred at the Command and Control Center and it all reinforces my bias against Linux and my love for Windows. Well, ok, maybe that's an exaggeration, but the good, the bad, and the ugly of both operating systems became very apparent to me recently.

October is when Linux Ubuntu get's its semi-annual upgrade. The normal routine for me is to download the iso copy, burn it to DVD, and then use the downloaded copy to create a virtual machine (VM). If all goes swell in the VM version I will replace any HDD versions I have and/or update USB memory stick versions. It's quite an elaborate ritual, I know, but it keeps my mind fresh on how to do all these things. Plus, it's a system of redundant backups so that I would never be without Internet access as long as there is a computer I can plug a USB stick into. If I'm on the road, I'll have the LTS version of Ubuntu in my laptop but also carry along a stick version. You can never be too redundant. LOL

Those of you who have created operating systems on a memory stick know it's not so difficult, if you have the right tools. Both Windows and Linux have software that is capable of burning an iso image to CD and then to a memory stick, not to mention the live CD for Ubuntu has it integrated into the desktop. What can go wrong? Ext4, FAT32, BIOS, UEFI, gparted, and the mind boggling collection of options on how to configure all that is what can go wrong. Take for example the well established and mature software called Unetbootin. This program is widely used to create bootable memory sticks for any Linux distribution out there, but the stick must be formatted in FAT32 before Ubuntu will even recognize it exists. Unfortunately, if Unetbootin is run from Windows, an NTFS file system has no clue about what to do with FAT32. So, it ignores it. It's the same problem using Rufus, the alternative to Unetbootin. So, at this point I was wondering why the hell am I using Windows to create a Linux OS anyway? Then gain … I know I did it a few times before.

Groping for solutions, I fired up the laptop version of Ubuntu and used the built in software to create a bootable USB memory stick onto which I can install … Ubuntu. That went well until it came time to install Ubuntu onto the stick. Like Windows, the memory stick was not recognized. Didn't matter if it was formatted in FAT32 or ext4, or degaussed using the Fermilab's particle accelerator which is not too far from here. After several frustrating attempts to do what I've done dozens of times before, I decided to start from scratch and do this one step at a time, trying to be rational every step of the way.

I took the USB memory stick out of the laptop and stuck it in the tower. Then I fired up gparted. The stick looked fine from there and I decided to repartition it to be entirely FAT32. This is a UEFI requirement and since that's all I have these days I decided that's the way to go. So click a few buttons and voila. The USB stick is formatted FAT32 … no wait. That wasn't the USB stick I just formatted. It was the hard drive with Windows 7 on it, /dev/sda1 and not /dev/sdc1. Gparted will show all the hardware that can be partitioned, but it does it one device at a time, and the default device it presents is the lowest in the pecking order. In my case that would be drive \:C with Windows.

Sure enough, when I tried to boot Windows, the bootloader said, “huh?” Oddly enough, I was able to run the Windows recovery disk and fix the MBR, which proved useless because there was nothing for the MBR to point to. Gparted took care of that. The long and the short of it is that I had made an image of my Windows 7 machine several weeks ago. Plus I have been backing up critical data at least once a week. Thus, after much wailing and gnashing of teeth I had Windows back. I'll tell you about my “learning experience” with backups in another thread, but let it suffice here to say that incremental backups should only be done at an enterprise level. Us desktop people should never do less than a full back up.

It took many days to reinstall the critical programs that I use more or less daily. Once I had all that working, I made another image of Windows 7. This was a clean and mean version now, and it is well worth preserving in case I ever need it again. Little did I know how soon I would be needing it. After about two weeks of convincing myself Windows is where I want it to be, I decided to go back to making a USB version of Ubuntu. That was a mistake because once again I tried to format a new stick by using gparted. Guess what I did FOR A SCOND TIME IN TWO WEEKS! This time recovering Windows 7 was a lot easier because I had an image that was only a couple days old. I was back to where I was earlier in the day in under 30 minutes. It takes a long time to copy 80GB of data.

I decided a few days ago that Linux is beautiful with all it's options and flexibility and free software, but it's not foolproof and it is extremely complicated. I should not complain because they tell you to be careful when you partition things and are the root user. I was not careful, obviously, but Linux is too flexible. It takes A LOT of pre-knowledge to do even the most simple of things. My last stab at creating a USB stick was to do it from the live CD copy of Ubuntu. Well … that worked flawlessly. It took about as long as it takes to recover Windows 7, but it copied and was bootable without error. The lesson learned here is to never ever attempt to make a bootable USB stick version of Ubuntu any other way than by using the integrated installer on the live CD. It is said that Linux is flexible enough to do it a number of other ways, but don't believe it. It's all a pack of lies to say it's easy and foolproof.
User avatar
Kellemora
Guardian Angel
Guardian Angel
Posts: 7494
Joined: 16 Feb 2015, 17:54

Re: Creating Bootable Memory Sticks

Post by Kellemora »

Poor Yogi

Almost every ISO I've ever downloaded I placed a copy on a blank USB stick, without making a CD or DVD, simply because it is faster. I've never reformatted a USB stick from how it came from the factory, so I assume they are NTFS since they work to save files from Windoze computers.

I don't usually go through the several steps to figure out what Drive Letter it was assigned on Windows machines, or on Linux for that matter, I just look under My Computer and can find it there much faster.
Doesn't matter if I'm using a Windows machine or Linux machine, ISOs are automagically bootable.

On Windows 8.1 or 10 you just use Windows Set-up and select USB as the destination.
It will ask whether to copy an ISO file or copy the Windows system.
On Linux you just use the Universal USB Installer and it does the rest.
It will ask if from an ISO or IMG file and the location and name of the file.

The only problem I found with Windoze is you must have the Product Key to even begin the USB install.

I'll admit, it has been a while since I've made a USB stick, my last one was Debian 8 done on a Linux machine. I used Pendrive's Universal USB Installer.

Since you used GParted, you do really have to be careful to make sure you have selected the proper drive.
The on-board main drive is the default drive. You must change drives to the destination drive to do anything with it.
If you change screens, it falls back to the default drive without warning. This is what trips many folks up.

I've made Windows System ISOs directly to DVDs for my Windows computers, namely our XP computers. You can always install back to the same computer with a new hard drive. But I've never got one to work yet if I tried moving to a different computer. Oh, it will boot up the DVD, but installing it to a different computer failed 100% of the time.
Doing the same thing with a Linux computer, once I had it working the way I wanted, then making an ISO to DVD has worked 90% of the time when trying to install to a different computer.
The only thing it saves is the time to reinstall all the programs you normally use. You still have to update all of your personal files since the time you made the ISO DVD. One of the reasons I only backup my Home folder now. I do not use a Home folder partition, I let each Distro keep its own Home folder. Saves a lot of headaches if you use different distro's.

Glad you are up and running again!
User avatar
yogi
Posts: 9978
Joined: 14 Feb 2015, 21:49

Re: Creating Bootable Memory Sticks

Post by yogi »

There is more than one way to skin a cat, plus you and I are skinning different kinds of cats. :mrgreen:

Believe it or not, I have never saved an iso download directly to a USB memory stick. The sticks are faster to "burn" depending on what kind of USB drivers you have verses your DVD burner. However, the sticks are a lot like you and I and lose their memory over time. DVD's will too, but not as quickly. Also, the computer you use must be able to boot from USB. Some computers need to be configured ahead of time to do that. Other computers simply can't - usually very old ones. I have a computer in my basement that doesn't even know what a USB port is. In any case, DVD's and CD's are a fraction of the price of a flash memory stick.

Most, if not all, the USB sticks I've bought new need to be formatted. I don't think they come with NTFS simply because there is a lot of FAT32 out in the wilds. UEFI requires FAT32 in fact. I might have tried using a few right out of the box but I don't recall if Windows required formatting. It all depends on what kind of memory firmware is supplied from the factory. When the intention is to install a working operating system to a USB stick, the correct bootloader software must be preloaded. That bootloader is not the same as an MBR on a hard drive nor does it resemble GRUB. It's unique to USB memory. Now with UEFI being standard in all computers made recently, the bootloader can be BIOS, UEFI, or both. Too bad if you pick the wrong one. You will get a "no operating system found" error regardless of how clean the actual software install is.

I'd be interested in knowing what you do to get an iso image to run simply by downloading to the memory stick. I've never been able to do that because of those required bootloader drivers. Memory sticks typically are not bootable out of the box for the same reason they (probably) are not formatted. You can be using that stick in any environment, and each environment has it's own special requirements. An iso image will run, but I have yet to be able to get one to boot without some kind of firmware installed on the same media ahead of time.

Your comment about Windows requiring a product key to burn a USB image is interesting because I never had to supply one. I can see them wanting it if you are trying to copy a version of Windows OS to the stick. They consider that to be a unique instance that needs to have a license. Same is true when making virtual machines of Windows. You need to get a license for each copy. They claim you can make one archive copy, and as my story of woe suggests I have done that successfully. But, when I restored the system from the archive copy, I had to re-activate Windows.

The only glitch that I can see possible when trying to restore a system from an archive copy of Windows has to do with disk drive size and partition formatting. The destination has to be the same format and of equal size, or larger. My blown away Windows 7 was a restored version from an archive copy, but the new SSD (the target for the restoration) was a lot larger than the original HDD from whence it came. I had no problems at all doing the recovery. Didn't have to activate it either, now that I think about it.

Gparted works exactly as you describe it, and I knew all that going into the game. It was simply carelessness on my part to not verify what partition I was pointing to. You also have to select which disk and partition thereupon to install Ubuntu's bootloader (GRUB). Otherwise it defaults to the lowest lettered drive, /dev/sda, which is generally where I keep windows. It's fine to boot Windows from Grub, gut there are some situations where Windows will not work properly if you don't use it's native MBR information. Making network backups, for example, must be done from the same drive upon which the bootloader is installed. You would not believe how long it took to figure that one out.

I don't use Linux enough to worry about backups, but I do keep a copy of my personal /home directory in a backup file on my NAS. That way I can hand copy all the useful information without having to bother with drivers or other system related information. I simply install a new OS and then copy my /home backup to it when appropriate. It's crude, but simple and works every time. :grin:
User avatar
Kellemora
Guardian Angel
Guardian Angel
Posts: 7494
Joined: 16 Feb 2015, 17:54

Re: Creating Bootable Memory Sticks

Post by Kellemora »

Hi Yogi

I just pulled a brand new 8 gig stick from my box of what was once a half dozen, now only three left.

It pops up on the desktop as a removable drive.
I checked the File System and it says MSDOS, which I assume is FAT32.

It has three pre-installed folders. Club-Application, San-Disk-Secure-Access, & System-Volume-Information.
Plus two executables for Windows computers, RunSanDiskSecureAccess_Win.exe, & RunClubSanDisk.exe
Plus two files, autorun.inf, & _disk_id.pod
Needless to say, since I'm running Linux, I cannot run those files, hi hi...
Within the second folder mentioned above are PDF files of Instructions for Windows, MAC, and Ipods.

In it's out of the box state, it is ready to create new folders or add new files. Works on the frau's Windows machine from XP to Win10 with no problem. And obviously on my Linux boxes with no problem.

IF you want to make this bootable so it holds a LIVE ISO, everything on it will be deleted, and the ISO download will install everything it needs to boot up and run. You just have to Select USB STICK instead of CD or DVD from the burner program.

Everyone says Unetbootin is the way to go, I've never used it. The last time I burned a copy of Ubuntu to a USB stick, the package to do so was part of the download I selected. The couple of times I did it manually, I think I used a PenDriveLinux program. CD/DVD burners like Brassario don't usually include the USB option.

It has been a while since I've done an ISO to bootable USB, and I honestly don't remember if there were any other steps involved. You know me, I look up how to do something, do it, then forget what I did within a few days.
User avatar
yogi
Posts: 9978
Joined: 14 Feb 2015, 21:49

Re: Creating Bootable Memory Sticks

Post by yogi »

FAT32 was invented to run DOS (both PCDOS and MSDOS), all of which was intended to replace CP/M that could be run off a 5.25' floppy. Gadzooks. I still can recall running WordStar in CP/M on my very first 8-BIT computer. Damn, i am so old. :xclaim: :xclaim:

I'm aware of the pre-installed software that comes on the SanDisk sticks. I'm pretty sure something similar is on every new flash drive these days. Most people are using them for for external file storage as opposed to me who use them primarily for portable operating systems. I typically would use a partitioning program to format the memory so that I can reclaim the space consumed by the pre-installed software. Unfortunately, when I do that it also wipes out any boot loaders which is why I run into problems creating bootable USB memory sticks.

Unetbootin has been around as long as I've been using Linux. You are right to point out that many people are in love with it, but I seldom have success using it. On those occasions when it actually does what it's purported to be capable of doing, it attaches it's own branded front end. I don't care for their blatant advertising, plus they seem to have their own set of bootloaders. The only consistent way to creating a bootable version of Ubuntu is to use their native install program on a live CD. I don't know what they do exactly, but it's a recognizable GRUB front end and the disk format and OS is all transparent. I guess I don't know enough about what's involved to use alternatives properly. But I now have current system images for when I decide to learn. :mrgreen:
User avatar
Kellemora
Guardian Angel
Guardian Angel
Posts: 7494
Joined: 16 Feb 2015, 17:54

Re: Creating Bootable Memory Sticks

Post by Kellemora »

Hi Yogi

As complicated as I make some things, when it came to making bootable USB sticks with a Live OS, I guess I never gave it much thought. The couple of programs I used did everything, not much different than creating a bootable DVD with the ISO of a live OS on it.

The first place I used to build computers for me, they gave me a USB stick which contained four different Linux distributions on it to try them out before they installed one on my new computer. After the first computer, I did all the rest myself. I do remember it did not work quite like an installer on USB stick did, nor did it work like a Live CD either.
I'm assuming they actually INSTALLED the OSs on the USB Stick, because they did not have an installer file to install them on your computer from the USB stick.
You had to set your bio's to boot from USB, which on my first machine was simply press F12 to get a screen which asked where to boot from. Use the down arrow to highlight the correct line, then hit enter. After the machine booted up, you got GRUB, which had a ten second delay instead of the 2 second delay on regular installs. Again, you used the down arrow to get to the OS you wanted to use, and hit enter.

I've read a few things which are slightly above my head, about making a Bootable USB stick first, then adding the ISO. I've never tried this myself. Seems to me, unless the ISO was already a Live Distro, it wouldn't work using it from a USB stick, because it is not installed. I've had a couple of USB sticks which contained IMG files instead of ISO files, but could never figure out how to use them, so reformatted the USB stick, and after that, it would no longer open on my desktop automagically. I had to go to the File Folder and open it from their like any other folder.

One thing I wanted to do, but never figured out how, was to make an ISO of an Operating System, once I had it set up with all the things I wanted on it, and before I started adding files and other things which change often.
For E.g., the OS, after I got it set the way I wanted it with panels and the things in the panels, and the Programs I normally use, once they were set up and working properly, before I used the machine for doing daily work.

My reason for wanting to do this was so I only had to set up one machine, then I could just install on the other machines and be good to go, except for the other machine may have different drivers for sound, video, etc. which leaves me up that proverbial creek without a paddle, hi hi...

Although everyone said use /home in its own partition. Doing that caused me all kinds of grief, since I use different OSs on the same machine. So I let the installer place /home in the OSs own partition, no more problems.
And to keep other problems from happening, all data I generate gets saved redundantly to internal data partitions and to external hard drives, both here in the office and down at the house. I can replace everything except my data, and data in and of itself does not take that much space, except for the amount I've amassed over the years.

I'm starting to look more closely at cloud storage. My little drop box folder has come in useful, and even with all I've put on there, mainly so I could move it to the frau's Windows 10 machine. I've not used 1/10th of the free space I have available. I'm told my worries about security issues are pretty much unfounded. I would probably win the lottery twice before a hacker stumbled into even one of my files. Unless I was world famous and they were seeking me out.

OK, I'm rambling, as usual.

Have a great day Yogi!
Post Reply