Core Joomla updates are important - never ignore available updates as this could lead to your site becoming insecure. Apply them where ever possible, but ensure that you don't overwrite any modification made to the Joomla core code. From Joomla version 2.5 your Joomla site will tell you in the admin area if there are new updates available.
Also keep your Joomla Addons up to date, many security breaches in Joomla sites are due to third party addons. From Joomla 2.5 many of these addons have automatic update notifications within Joomla admin.
Before applying updates - ensure you make a site backup.
Need help in ensuring your site has the correct updates applied? - contact us for assistance.