U may want to get further away...fertilizer fires tend to turn into huge fertilizer explosions. Remember the port in the middle east of a few years ago. Balooweeee!
Texas City, Texas back in the 50's(?). That started off as a fertilizer fire on a boat. Baloooweeee Huge explosion ensued.
Three miles is probably safe...depending on the size of the fertilizer stock pile.
Be safe.