thredds.inventory

Class MFileCollectionManager

java.lang.Object

thredds.inventory.CollectionManagerAbstract

thredds.inventory.MFileCollectionManager

All Implemented Interfaces:

CollectionManager

Direct Known Subclasses:

CatalogCollectionManager, TimePartitionCollection

@ThreadSafe
public class MFileCollectionManager
extends CollectionManagerAbstract

Manage Collections of MFiles. Used in:

• replaces older version in ncml.Aggregation
• ucar.nc2.ft.point.collection.TimedCollectionImpl (ucar.nc2.ft.point.collection.CompositeDatasetFactory)
• ucar.nc2.ft.fmrc.Fmrc
• ucar.nc2.grib.GribCollection

we need to be thread safe, for InvDatasetFeatureCollection

Since:

Jul 8, 2009

Author:

caron

Nested Class Summary

Nested classes/interfaces inherited from interface thredds.inventory.CollectionManager

CollectionManager.ChangeChecker, CollectionManager.Force, CollectionManager.TriggerEvent, CollectionManager.TriggerListener, CollectionManager.TriggerType

Field Summary

Fields

Modifier and Type	Field and Description
static String	CATALOG
protected FeatureCollectionConfig	config
protected DateExtractor	dateExtractor
protected CalendarDate	startPartition

Fields inherited from class thredds.inventory.CollectionManagerAbstract

auxInfo, changeChecker, collectionName, logger, protoChoice, recheck

Constructor Summary

Constructors

Modifier	Constructor and Description
	MFileCollectionManager(FeatureCollectionConfig config, Formatter errlog, org.slf4j.Logger logger)
protected	MFileCollectionManager(String name, org.slf4j.Logger logger)
	MFileCollectionManager(String name, MCollection mc, CalendarDate startPartition, org.slf4j.Logger logger)
	MFileCollectionManager(String name, String spec, Formatter errlog, org.slf4j.Logger logger)

Method Summary

Methods

Modifier and Type	Method and Description
void	addDirectoryScan(String dirName, String suffix, String regexpPatternString, String subdirsS, String olderS, Object auxInfo) Add a directory scan to the collection
CalendarDate	extractRunDate(MFile mfile) Use the date extractor to extract the date from the filename.
static MController	getController()
Iterable<MFile>	getFiles() Get the current collection of MFile.
long	getLastScanned() Get the last time scanned
long	getOlderThanFilterInMSecs() The "olderThan" amount in seconds.
String	getRoot() Get common root directory of all MFiles in the collection - may be null
CalendarDate	getStartCollection() The starting date of the collection.
boolean	hasDateExtractor() Does this CollectionManager have the ability to extract a date from the MFile ?
protected boolean	hasScans()
boolean	isScanNeeded() Compute if synchronous scan is needed.
static MFileCollectionManager	open(String collectionName, String olderThan, Formatter errlog)
static MFileCollectionManager	openWithRecheck(String collectionName, String recheckS)
protected void	reallyScan(Map<String,MFile> map)
boolean	scan(boolean sendEvent) Scan the collection.
void	scanDebug(Formatter f) Do not use
boolean	scanIfNeeded() If isScanNeeded(), do a scan.
static void	setController(MController _controller) Set the MController used by scan.
void	setDateExtractor(DateExtractor dateExtractor)
void	setFiles(Iterable<MFile> files) dcm must be updated when index is read in
String	toString()

Methods inherited from class thredds.inventory.CollectionManagerAbstract

addEventListener, cleanName, close, createListenerManager, deleteOld, enableMetadataManager, getAuxInfo, getCollectionName, getFilenames, getLatestFile, getMetadata, getProtoIndex, getRecheck, isStatic, putAuxInfo, putMetadata, removeEventListener, resetProto, setChangeChecker, setStatic, updateNocheck

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CATALOG

public static final String CATALOG

See Also:

Constant Field Values

dateExtractor

protected DateExtractor dateExtractor

startPartition

protected CalendarDate startPartition

config

protected FeatureCollectionConfig config

Constructor Detail

MFileCollectionManager

public MFileCollectionManager(FeatureCollectionConfig config,
                      Formatter errlog,
                      org.slf4j.Logger logger)

MFileCollectionManager

protected MFileCollectionManager(String name,
                      org.slf4j.Logger logger)

MFileCollectionManager

public MFileCollectionManager(String name,
                      String spec,
                      Formatter errlog,
                      org.slf4j.Logger logger)

MFileCollectionManager

public MFileCollectionManager(String name,
                      MCollection mc,
                      CalendarDate startPartition,
                      org.slf4j.Logger logger)

Method Detail

setController

public static void setController(MController _controller)

Set the MController used by scan. Defaults to thredds.filesystem.ControllerOS() if not set.

Parameters:

_controller - use this MController

getController

public static MController getController()

open

public static MFileCollectionManager open(String collectionName,
                          String olderThan,
                          Formatter errlog)
                                   throws IOException

Throws:

IOException

openWithRecheck

public static MFileCollectionManager openWithRecheck(String collectionName,
                                     String recheckS)

getStartCollection

public CalendarDate getStartCollection()

Description copied from interface: CollectionManager

The starting date of the collection. Only call if hasDateExtractor() == true.

Returns:

starting date of the collection

setDateExtractor

public void setDateExtractor(DateExtractor dateExtractor)

addDirectoryScan

public void addDirectoryScan(String dirName,
                    String suffix,
                    String regexpPatternString,
                    String subdirsS,
                    String olderS,
                    Object auxInfo)

Add a directory scan to the collection

Parameters:

dirName - scan this directory

suffix - require this suffix (overriddden by regexp), may be null

regexpPatternString - if present, use this reqular expression to filter files , may be null

subdirsS - if "true", descend into subdirectories, may be null

olderS - udunit time unit - files must be older than this amount of time (now - lastModified > olderTime), may be null // * @param dateFormatString dateFormatMark string, may be null

auxInfo - attach this object to any MFile found by this scan

getRoot

public String getRoot()

Description copied from interface: CollectionManager

Get common root directory of all MFiles in the collection - may be null

Returns:

root directory name, or null.

getOlderThanFilterInMSecs

public long getOlderThanFilterInMSecs()

Description copied from interface: CollectionManager

The "olderThan" amount in seconds. Files are excluded if they have been modified within this amount of time. However once in the collection they are not removed. Really this is handled by the manager, but this is exposed so that others (eg DatasetScan) can be consistent.

Specified by:

getOlderThanFilterInMSecs in interface CollectionManager

Overrides:

getOlderThanFilterInMSecs in class CollectionManagerAbstract

Returns:

olderThan" amount in seconds, or < 0 to mean this filter is not present

getLastScanned

public long getLastScanned()

Description copied from interface: CollectionManager

Get the last time scanned

Returns:

msecs since 1970

scanIfNeeded

public boolean scanIfNeeded()
                     throws IOException

Description copied from interface: CollectionManager

If isScanNeeded(), do a scan.

Returns:

true if scan was done, and anything changed.

Throws:

IOException - on io error

hasScans

protected boolean hasScans()

isScanNeeded

public boolean isScanNeeded()

Compute if synchronous scan is needed. True if recheck is true and enough time has elapsed.

Returns:

true if rescan is needed

scanDebug

public void scanDebug(Formatter f)
               throws IOException

Do not use

Throws:

IOException

scan

public boolean scan(boolean sendEvent)
             throws IOException

Description copied from interface: CollectionManager

Scan the collection. Files may have been deleted or added since last time. If the MFile already exists in the current list, leave it in the list. If anything changes, send TriggerEvent(TriggerType.update) and return true Get the results from getFiles()

Returns:

true if anything actually changed.

Throws:

IOException - on I/O error

setFiles

public void setFiles(Iterable<MFile> files)

Description copied from interface: CollectionManager

dcm must be updated when index is read in

getFiles

public Iterable<MFile> getFiles()

Description copied from interface: CollectionManager

Get the current collection of MFile. You must call scan() first - this does not call scan(). if hasDateExtractor() == true, these will be sorted by Date, otherwise by path.

Returns:

current collection of MFile as an Iterable. May be empty, not null.

extractRunDate

public CalendarDate extractRunDate(MFile mfile)

Description copied from interface: CollectionManager

Use the date extractor to extract the date from the filename. Only call if hasDateExtractor() == true.

Parameters:

mfile - extract from here

Returns:

Date, or null if none

hasDateExtractor

public boolean hasDateExtractor()

Description copied from interface: CollectionManager

Does this CollectionManager have the ability to extract a date from the MFile ?

Returns:

true if CollectionManager has a DateExtractor

reallyScan

protected void reallyScan(Map<String,MFile> map)
                   throws IOException

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object