Over the last decade, there has been no discernible reduction in Invasive Pneumococcal Disease (IPD) amongst Indigenous adults in the Northern Territory (NT) of Australia, despite increasing vaccination coverage. We examined the utility of two common methods, the screening method and the indirect method, to determine the 23-valent pneumococcal polysaccharide vaccine effectiveness (VE) in prevention of IPD amongst Indigenous adults in this setting. VE was calculated for the period 2001-2005 across two distinct geographical areas where the disease burden was known to differ. VE against vaccine-type IPD was 3.4% (95% CI -43, 35) for the NT. However, population vaccination coverage varied widely according to geographical region and where this was within the range appropriate for the use of the screening method, VE was within the expected range (67.2%, 95% CI 47, 80). VE according to the indirect cohort appeared unreliable in this setting due to the analysis being based on a very limited number of non-vaccine-type IPD cases. Surveillance based estimates of VE such as these need to be considered with caution, but the results suggest failure to vaccinate is the most likely reason vaccine-type IPD has not reduced in this setting.