Weather can, if you let it, make or break your trip. Don't be caught off guard! Check the annual average weather for the place you will be visiting for the time you will be visiting (but take it with a grain of salt - my last trip I was expecting high teens to low 20's (celsius) and it wound up being more in the 30's). I usually bring at least a (ultra)light rain jacket with me if there's going to be a chance of rain, or a heavier jacket if it's going to be cold. It is possible to enjoy the (rain/snow/heat/cold) if you are dressed for it and prepared! On a recent trip to Iceland, we hiked up to Skogafoss and past during a wind/rain storm. We didn't go as far as we would have liked because it was starting to feel almost dangerous with the extreme winds, but we didn't let it stop us from enjoying the views nonetheless.
Weatherbase.com provides a great 'vacation finder' at the bottom of their home page that allows you to select the month you'd like to travel and then a minimum and maximum temperature range and provides you with a list of places based on that criteria. This link for example shows you the average temperature as well as the average high, low, rainy days, rainfall, and snowfall for cities in New Zealand in January.
Hurricanes and other natural disasters seem to be occurring more frequently with the changing weather patterns we've been having. Do not allow this to be a deterrent to your trip, but be aware that these kinds of things can happen, and they can be more likely to occur during the 'off season' (hence one of the reasons of there being an off season heh). The NOAA has released an interactive hurricane search tool here which allows you to search the desired location and will bring up any hurricanes over the last century or so. HurricaneScience has released a 'During a Hurricane: How to be Safe' list here. The main advice is to "stay inside and away from windows"!