ucar.nc2.util.cache

Class FileCacheARC

java.lang.Object

ucar.nc2.util.cache.FileCacheARC

All Implemented Interfaces:

FileCacheIF

@ThreadSafe
public class FileCacheARC
extends Object
implements FileCacheIF

Keep cache of open FileCacheable objects, for example NetcdfFile. The FileCacheable object typically contains a RandomAccessFile object that wraps a system resource like a file handle. These are left open when the FileCacheable is in the cache. The maximum number of these is bounded, though not strictly. A cleanup routine reduces cache size to a minimum number. This cleanup is called periodically in a background thread, and also when the maximum cache size is reached.

1. The FileCacheable object must not be modified.
2. The hashKey must uniquely define the FileCacheable object.
3. The location must be usable in a FileFactory.open().
4. If the FileCacheable is acquired from the cache (ie already open), getLastModified() is used to see if it has changed, and is discarded if so.
5. Make sure you call shutdown() when exiting the program, in order to shut down the cleanup thread.

Normal usage is through the NetcdfDataset interface:

 NetcdfDataset.initNetcdfFileCache(...); // on application startup
 ...
 NetcdfFile ncfile = null;
 try {
   ncfile = NetcdfDataset.acquireFile(location, cancelTask);
   ...
 } finally {
   if (ncfile != null) ncfile.close();
 }
 ...
 NetcdfDataset.shutdown();  // when terminating the application
 

All methods are thread safe. Cleanup is done automatically in a background thread, using LRU algorithm.

Since:

10/28/2014

Author:

caron

Field Summary

Fields

Modifier and Type	Field and Description
protected ConcurrentHashMap<Object,ucar.nc2.util.cache.FileCacheARC.CacheElement>	cache
protected static org.slf4j.Logger	cacheLog
protected ConcurrentHashMap<Integer,ucar.nc2.util.cache.FileCacheARC.CacheElement.CacheFile>	files
protected int	hardLimit
protected AtomicInteger	hits
protected static org.slf4j.Logger	log
protected int	minElements
protected AtomicInteger	miss
protected String	name
protected int	period
protected ConcurrentSkipListMap<ucar.nc2.util.cache.FileCacheARC.CacheElement,ucar.nc2.util.cache.FileCacheARC.CacheElement>	shadowCache
protected int	softLimit
protected ConcurrentHashMap<Object,ucar.nc2.util.cache.FileCacheARC.Tracker>	track

Constructor Summary

Constructors

Constructor and Description
FileCacheARC(String name, int minElementsInMemory, int softLimit, int hardLimit, int period) Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
FileCacheable	acquire(FileFactory factory, Object hashKey, String location, int buffer_size, CancelTask cancelTask, Object spiObject) Acquire a FileCacheable from the cache, and lock it so no one else can use it.
FileCacheable	acquire(FileFactory factory, String location) Acquire a FileCacheable, and lock it so no one else can use it.
void	clearCache(boolean force) Remove all cache entries.
void	disable() Disable the cache, and force release all files.
void	enable() Enable the cache, with the current set of parameters.
String	getInfo(FileCacheable ncfile)
void	release(FileCacheable ncfile) Release the file.
void	resetTracking()
List<String>	showCache()
void	showCache(Formatter format) Show individual cache entries, add to formatter.
void	showStats(Formatter format) Add stat report (hits, misses, etc) to formatter.
void	showTracking(Formatter format)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

cacheLog

protected static final org.slf4j.Logger cacheLog

name

protected String name

softLimit

protected final int softLimit

minElements

protected final int minElements

hardLimit

protected final int hardLimit

period

protected final int period

shadowCache

protected final ConcurrentSkipListMap<ucar.nc2.util.cache.FileCacheARC.CacheElement,ucar.nc2.util.cache.FileCacheARC.CacheElement> shadowCache

cache

protected final ConcurrentHashMap<Object,ucar.nc2.util.cache.FileCacheARC.CacheElement> cache

files

protected final ConcurrentHashMap<Integer,ucar.nc2.util.cache.FileCacheARC.CacheElement.CacheFile> files

hits

protected final AtomicInteger hits

miss

protected final AtomicInteger miss

track

protected ConcurrentHashMap<Object,ucar.nc2.util.cache.FileCacheARC.Tracker> track

Constructor Detail

FileCacheARC

public FileCacheARC(String name,
            int minElementsInMemory,
            int softLimit,
            int hardLimit,
            int period)

Constructor.

Parameters:

name - of file cache

minElementsInMemory - keep this number in the cache

softLimit - trigger a cleanup if it goes over this number.

hardLimit - if > 0, never allow more than this many elements. This causes a cleanup to be done in the calling thread.

period - if > 0, do periodic cleanups every this number of seconds.

Method Detail

disable

public void disable()

Disable the cache, and force release all files. You must still call shutdown() before exiting the application.

Specified by:

disable in interface FileCacheIF

enable

public void enable()

Enable the cache, with the current set of parameters.

Specified by:

enable in interface FileCacheIF

acquire

public FileCacheable acquire(FileFactory factory,
                    String location)
                      throws IOException

Acquire a FileCacheable, and lock it so no one else can use it. call FileCacheable.close() when done.

Specified by:

acquire in interface FileCacheIF

Parameters:

factory - use this factory to open the file; may not be null

location - file location, also used as the cache name, will be passed to the NetcdfFileFactory

Returns:

NetcdfFile corresponding to location.

Throws:

IOException - on error

acquire

public FileCacheable acquire(FileFactory factory,
                    Object hashKey,
                    String location,
                    int buffer_size,
                    CancelTask cancelTask,
                    Object spiObject)
                      throws IOException

Acquire a FileCacheable from the cache, and lock it so no one else can use it. If not already in cache, open it with FileFactory, and put in cache.

Call FileCacheable.close() when done, (rather than FileCacheIF.release() directly) and the file is then released instead of closed.

If cache size goes over maxElement, then immediately (actually in 100 msec) schedule a cleanup in a background thread. This means that the cache should never get much larger than maxElement, unless you have them all locked.

Specified by:

acquire in interface FileCacheIF

Parameters:

factory - use this factory to open the file if not in the cache; may not be null

hashKey - unique key for this file. If null, the location will be used

location - file location, may also used as the cache name, will be passed to the NetcdfFileFactory

buffer_size - RandomAccessFile buffer size, if <= 0, use default size

cancelTask - user can cancel, ok to be null.

spiObject - sent to iosp.setSpecial() if not null

Returns:

FileCacheable corresponding to location.

Throws:

IOException - on error

release

public void release(FileCacheable ncfile)
             throws IOException

Release the file. This unlocks it, updates its lastAccessed date. FileCacheable.close() needs to call this instead of actually closing.

Specified by:

release in interface FileCacheIF

Parameters:

ncfile - release the lock on this FileCacheable object.

Throws:

IOException - if file not in cache.

getInfo

public String getInfo(FileCacheable ncfile)
               throws IOException

Throws:

IOException

clearCache

public void clearCache(boolean force)

Remove all cache entries.

Specified by:

clearCache in interface FileCacheIF

Parameters:

force - if true, remove them even if they are currently locked.

showCache

public void showCache(Formatter format)

Show individual cache entries, add to formatter.

Specified by:

showCache in interface FileCacheIF

Parameters:

format - add to this

showCache

public List<String> showCache()

Specified by:

showCache in interface FileCacheIF

showStats

public void showStats(Formatter format)

Add stat report (hits, misses, etc) to formatter.

Specified by:

showStats in interface FileCacheIF

Parameters:

format - add to this

showTracking

public void showTracking(Formatter format)

Specified by:

showTracking in interface FileCacheIF

resetTracking

public void resetTracking()

Specified by:

resetTracking in interface FileCacheIF