PAMI.periodicFrequentPattern.basic package

Submodules

PAMI.periodicFrequentPattern.basic.PFECLAT module

class PAMI.periodicFrequentPattern.basic.PFECLAT.PFECLAT(iFile, minSup, maxPer, sep='\t')[source]

Bases: _periodicFrequentPatterns

About this algorithm

Description:

PFECLAT is the fundamental approach to mine the periodic-frequent patterns.

Reference:

P. Ravikumar, P.Likhitha, R. Uday kiran, Y. Watanobe, and Koji Zettsu, “Towards efficient discovery of periodic-frequent patterns in columnar temporal databases”, 2021 IEA/AIE.

Parameters:

  • iFile (str or URL or dataFrame) – Name of the Input file to mine complete set of frequent patterns.

  • oFile (str) – Name of the output file to store complete set of frequent patterns.

  • minSup (int or float or str) – The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float.

  • maxPer (int or float or str) – The user can specify maxPer either in count or proportion of database size. It controls the maximum number of transactions in which any two items within a pattern can reappear.

  • sep (str) – This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:

  • startTime (float) – To record the start time of the mining process.

  • endTime (float) – To record the completion time of the mining process.

  • finalPatterns (dict) – Storing the complete set of patterns in a dictionary variable.

  • memoryUSS (float) – To store the total amount of USS memory consumed by the program.

  • memoryRSS (float) – To store the total amount of RSS memory consumed by the program.

  • Database (list) – To store the transactions of a database in list.

  • mapSupport (Dictionary) – To maintain the information of item and their frequency.

  • lno (int) – It represents the total no of transactions

  • tree (class) – it represents the Tree class.

  • itemSetCount (int) – it represents the total no of patterns.

  • tidList (dict) – stores the timestamps of an item.

  • hashing (dict) – stores the patterns with their support to check for the closed property.

Methods:

  • mine() – Mining process will start from here.

  • getPatterns() – Complete set of patterns will be retrieved with this function.

  • save(oFile) – Complete set of periodic-frequent patterns will be loaded in to a output file.

  • getPatternsAsDataFrame() – Complete set of periodic-frequent patterns will be loaded in to a dataframe.

  • getMemoryUSS() – Total amount of USS memory consumed by the mining process will be retrieved from this function.

  • getMemoryRSS() – Total amount of RSS memory consumed by the mining process will be retrieved from this function.

  • getRuntime() – Total amount of runtime taken by the mining process will be retrieved from this function.

  • creatingOneItemSets() – Scan the database and store the items with their timestamps which are periodic frequent.

  • getPeriodAndSupport() – Calculates the support and period for a list of timestamps.

  • Generation() – Used to implement prefix class equivalence method to generate the periodic patterns recursively

Execution methods

Terminal command

Format:

(.venv) $ python3 PFECLAT.py <inputFile> <outputFile> <minSup> <maxPer>

Example usage:

(.venv) $ python3 PFECLAT.py sampleDB.txt patterns.txt 10.0 20.0

Note

minSup will be considered in percentage of database transactions

Calling from a python program

from PAMI.periodicFrequentPattern.basic import PFECLAT as alg

iFile = 'sampleDB.txt'

minSup = 10  # can also be specified between 0 and 1

maxPer = 20 # can also be specified between 0 and 1

obj = alg.PFECLAT(iFile, minSup, maxPer)

obj.mine()

periodicFrequentPatterns = obj.getPatterns()

print("Total number of Periodic Frequent Patterns:", len(periodicFrequentPatterns))

obj.save("periodicFrequentPatterns")

Df = obj.getPatternsAsDataFrame()

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits:

The complete program was written by P. Likhitha and revised by Tarun Sreepada under the supervision of Professor Rage Uday Kiran.

getMemoryRSS() → float[source]

Total amount of RSS memory consumed by the mining process will be retrieved from this function

Returns:

returning RSS memory consumed by the mining process

Return type:

float

getMemoryUSS() → float[source]

Total amount of USS memory consumed by the mining process will be retrieved from this function

Returns:

returning USS memory consumed by the mining process

Return type:

float

getPatterns() → dict[source]

Function to send the set of periodic-frequent patterns after completion of the mining process

Returns:

returning periodic-frequent patterns

Return type:

dict

getPatternsAsDataFrame() → DataFrame[source]

Storing final periodic-frequent patterns in a dataframe

Returns:

returning periodic-frequent patterns in a dataframe

Return type:

pd.DataFrame

getRuntime() → float[source]

Calculating the total amount of runtime taken by the mining process

Returns:

returning total amount of runtime taken by the mining process

Return type:

float

mine() → None[source]

Mining process will start from this function :return: None

printResults() → None[source]

This function is used to print the results :return: None

save(outFile: str) → None[source]

Complete set of periodic-frequent patterns will be loaded in to a output file

Parameters:

outFile (csv file) – name of the output file

Returns:

None

startMine() → None[source]

Code for the mining process will start from this function

PAMI.periodicFrequentPattern.basic.PFPGrowth module

class PAMI.periodicFrequentPattern.basic.PFPGrowth.PFPGrowth(iFile, minSup, maxPer, sep='\t')[source]

Bases: _periodicFrequentPatterns

About this algorithm

Description:

PFPGrowth is one of the fundamental algorithm to discover periodic-frequent patterns in a transactional database.

Reference:

Syed Khairuzzaman Tanbeer, Chowdhury Farhan, Byeong-Soo Jeong, and Young-Koo Lee, “Discovering Periodic-Frequent Patterns in Transactional Databases”, PAKDD 2009, https://doi.org/10.1007/978-3-642-01307-2_24

Parameters:

  • iFile (str or URL or dataFrame) – Name of the Input file to mine complete set of frequent patterns.

  • oFile (str) – Name of the output file to store complete set of frequent patterns.

  • minSup (int or float or str) – The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float.

  • maxPer (int or float or str) – The user can specify maxPer either in count or proportion of database size. It controls the maximum number of transactions in which any two items within a pattern can reappear.

  • sep (str) – This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:

  • startTime (float) – To record the start time of the mining process.

  • endTime (float) – To record the completion time of the mining process.

  • finalPatterns (dict) – Storing the complete set of patterns in a dictionary variable.

  • memoryUSS (float) – To store the total amount of USS memory consumed by the program.

  • memoryRSS (float) – To store the total amount of RSS memory consumed by the program.

  • Database (list) – To store the transactions of a database in list.

  • mapSupport (Dictionary) – To maintain the information of item and their frequency.

  • lno (int) – It represents the total no of transactions

  • tree (class) – it represents the Tree class.

  • itemSetCount (int) – it represents the total no of patterns.

Methods:

  • mine() – Mining process will start from here.

  • getPatterns() – Complete set of patterns will be retrieved with this function.

  • save(oFile) – Complete set of periodic-frequent patterns will be loaded in to a output file.

  • getPatternsAsDataFrame() – Complete set of periodic-frequent patterns will be loaded in to a dataframe.

  • getMemoryUSS() – Total amount of USS memory consumed by the mining process will be retrieved from this function.

  • getMemoryRSS() – Total amount of RSS memory consumed by the mining process will be retrieved from this function.

  • getRuntime() – Total amount of runtime taken by the mining process will be retrieved from this function.

  • creatingItemSets(fileName) – Scans the dataset and stores in a list format.

  • PeriodicFrequentOneItem() – Extracts the one-periodic-frequent patterns from database.

  • updateDatabases() – Update the database by removing aperiodic items and sort the Database by item decreased support.

  • buildTree() – After updating the Database, remaining items will be added into the tree by setting root node as null.

  • convert() – This methos is used to convert the user specified value.

Execution methods

Terminal command

Format:

(.venv) $ python3 PFPGrowth.py <inputFile> <outputFile> <minSup> <maxPer>

Example usage:

(.venv) $ python3 PFPGrowth.py sampleDB.txt patterns.txt 10.0 20.0

Note

minSup will be considered in percentage of database transactions

Calling from a python program

from PAMI.periodicFrequentPattern.basic import PFPGrowth as alg

iFile = 'sampleDB.txt'

minSup = 10  # can also be specified between 0 and 1

maxPer = 20 # can also be specified between 0 and 1

obj = alg.PFPGrowth(iFile, minSup, maxPer)

obj.mine()

periodicFrequentPatterns = obj.getPatterns()

print("Total number of Periodic Frequent Patterns:", len(periodicFrequentPatterns))

obj.save("periodicFrequentPatterns")

Df = obj.getPatternsAsDataFrame()

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits

The complete program was written by P. Likhitha and revised by Tarun Sreepada under the supervision of Professor Rage Uday Kiran.

getMemoryRSS() → float[source]

Total amount of RSS memory consumed by the mining process will be retrieved from this function

Returns:

returning RSS memory consumed by the mining process

Return type:

float

getMemoryUSS() → float[source]

Total amount of USS memory consumed by the mining process will be retrieved from this function

Returns:

returning USS memory consumed by the mining process

Return type:

float

getPatterns() → Dict[str, Tuple[int, int]][source]

Function to send the set of periodic-frequent patterns after completion of the mining process

Returns:

returning periodic-frequent patterns

Return type:

dict

getPatternsAsDataFrame() → DataFrame[source]

Storing final periodic-frequent patterns in a dataframe

Returns:

returning periodic-frequent patterns in a dataframe

Return type:

pd.DataFrame

getRuntime() → float[source]

Calculating the total amount of runtime taken by the mining process

Returns:

returning total amount of runtime taken by the mining process

Return type:

float

mine() → None[source]

Mining process will start from this function

Returns:

None

printResults() → None[source]

This function is used to print the results

Returns:

None

save(outFile: str) → None[source]

Complete set of periodic-frequent patterns will be loaded in to an output file

Parameters:

outFile (csv file) – name of the output file

Returns:

None

startMine() → None[source]

Code for the mining process will start from this function

PAMI.periodicFrequentPattern.basic.PFPGrowthPlus module

class PAMI.periodicFrequentPattern.basic.PFPGrowthPlus.PFPGrowthPlus(iFile, minSup, maxPer, sep='\t')[source]

Bases: _periodicFrequentPatterns

Description:

PFPGrowthPlus is fundamental and improved version of PFPGrowth algorithm to discover periodic-frequent patterns in temporal database. It uses greedy approach to discover effectively

Reference:

R. UdayKiran, MasaruKitsuregawa, and P. KrishnaReddyd, “Efficient discovery of periodic-frequent patterns in very large databases,” Journal of Systems and Software February 2016 https://doi.org/10.1016/j.jss.2015.10.035

param iFile:

str : Name of the Input file to mine complete set of periodic frequent pattern’s

param oFile:

str : Name of the output file to store complete set of periodic frequent pattern’s

param minSup:

str: Controls the minimum number of transactions in which every item must appear in a database.

param maxPer:

str: Controls the maximum number of transactions in which any two items within a pattern can reappear.

param sep:

str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:
iFilefile

Name of the Input file or path of the input file

oFilefile

Name of the output file or path of the output file

minSupint or float or str

The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float. Example: minSup=10 will be treated as integer, while minSup=10.0 will be treated as float

maxPerint or float or str

The user can specify maxPer either in count or proportion of database size. If the program detects the data type of maxPer is integer, then it treats maxPer is expressed in count. Otherwise, it will be treated as float. Example: maxPer=10 will be treated as integer, while maxPer=10.0 will be treated as float

sepstr

This variable is used to distinguish items from one another in a transaction. The default seperator is tab space or . However, the users can override their default separator.

memoryUSSfloat

To store the total amount of USS memory consumed by the program

memoryRSSfloat

To store the total amount of RSS memory consumed by the program

startTime:float

To record the start time of the mining process

endTime:float

To record the completion time of the mining process

Databaselist

To store the transactions of a database in list

mapSupportDictionary

To maintain the information of item and their frequency

lnoint

it represents the total no of transaction

treeclass

it represents the Tree class

itemSetCountint

it represents the total no of patterns

finalPatternsdict

it represents to store the patterns

Methods:
mine()

Mining process will start from here

getPatterns()

Complete set of patterns will be retrieved with this function

save(oFile)

Complete set of periodic-frequent patterns will be loaded in to a output file

getPatternsAsDataFrame()

Complete set of periodic-frequent patterns will be loaded in to a dataframe

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function

getRuntime()

Total amount of runtime taken by the mining process will be retrieved from this function

check(line)

To check the delimiter used in the user input file

creatingItemSets(fileName)

Scans the dataset or dataframes and stores in list format

PeriodicFrequentOneItem()

Extracts the one-periodic-frequent patterns from Databases

updateDatabases()

update the Databases by removing aperiodic items and sort the Database by item decreased support

buildTree()

after updating the Databases ar added into the tree by setting root node as null

mine()

the main method to run the program

Methods to execute code on terminal

Format:

(.venv) $ python3 PFPGrowthPlus.py <inputFile> <outputFile> <minSup> <maxPer>

Example:

(.venv) $ python3 PFPGrowthPlus.py sampleTDB.txt patterns.txt 0.3 0.4


        .. note:: minSup will be considered in percentage of database transactions

Importing this algorithm into a python program

from PAMI.periodicFrequentPattern.basic import PFPGorwthPlus as alg

obj = alg.PFPGrowthPlus("../basic/sampleTDB.txt", "2", "6")

obj.mine()

periodicFrequentPatterns = obj.getPatterns()

print("Total number of Periodic Frequent Patterns:", len(periodicFrequentPatterns))

obj.save("patterns")

Df = obj.getPatternsAsDataFrame()

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

The complete program was written by P.Likhitha under the supervision of Professor Rage Uday Kiran.

getMemoryRSS() → float[source]

Total amount of RSS memory consumed by the mining process will be retrieved from this function

Returns:

returning RSS memory consumed by the mining process

Return type:

float

getMemoryUSS() → float[source]

Total amount of USS memory consumed by the mining process will be retrieved from this function

Returns:

returning USS memory consumed by the mining process

Return type:

float

getPatterns() → Dict[str, Tuple[int, int]][source]

Function to send the set of periodic-frequent patterns after completion of the mining process

Returns:

returning periodic-frequent patterns

Return type:

dict

getPatternsAsDataFrame() → DataFrame[source]

Storing final periodic-frequent patterns in a dataframe

Returns:

returning periodic-frequent patterns in a dataframe

Return type:

pd.DataFrame

getRuntime() → float[source]

Calculating the total amount of runtime taken by the mining process

Returns:

returning total amount of runtime taken by the mining process

Return type:

float

mine() → None[source]

Main method where the patterns are mined by constructing tree. :return: None

printResults() → None[source]

This function is used to print the results :return: None

save(outFile: str) → None[source]

Complete set of periodic-frequent patterns will be loaded in to an output file

Parameters:

outFile (csv file) – name of the output file

Returns:

None

startMine() → None[source]

Main method where the patterns are mined by constructing tree. :return: None

PAMI.periodicFrequentPattern.basic.PFPMC module

class PAMI.periodicFrequentPattern.basic.PFPMC.PFPMC(iFile, minSup, maxPer, sep='\t')[source]

Bases: _periodicFrequentPatterns

Description:

PFPMC is the fundamental approach to mine the periodic-frequent patterns.

Reference:

(has to be added)

Parameters:

  • iFile – str : Name of the Input file to mine complete set of periodic frequent pattern’s

  • oFile – str : Name of the output file to store complete set of periodic frequent pattern’s

  • minSup – str: Controls the minimum number of transactions in which every item must appear in a database.

  • maxPer – str: Controls the maximum number of transactions in which any two items within a pattern can reappear.

  • sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:
iFilefile

Name of the Input file or path of the input file

oFilefile

Name of the output file or path of the output file

minSupint or float or str

The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float. Example: minSup=10 will be treated as integer, while minSup=10.0 will be treated as float

maxPerint or float or str

The user can specify maxPer either in count or proportion of database size. If the program detects the data type of maxPer is integer, then it treats maxPer is expressed in count. Otherwise, it will be treated as float. Example: maxPer=10 will be treated as integer, while maxPer=10.0 will be treated as float

sepstr

This variable is used to distinguish items from one another in a transaction. The default seperator is tab space or . However, the users can override their default separator.

memoryUSSfloat

To store the total amount of USS memory consumed by the program

memoryRSSfloat

To store the total amount of RSS memory consumed by the program

startTime:float

To record the start time of the mining process

endTime:float

To record the completion time of the mining process

Databaselist

To store the transactions of a database in list

mapSupportDictionary

To maintain the information of item and their frequency

lnoint

it represents the total no of transactions

treeclass

it represents the Tree class

itemSetCountint

it represents the total no of patterns

finalPatternsdict

it represents to store the patterns

tidListdict

stores the timestamps of an item

hashingdict

stores the patterns with their support to check for the closed property

Methods:
mine()

Mining process will start from here

getPatterns()

Complete set of patterns will be retrieved with this function

save(oFile)

Complete set of periodic-frequent patterns will be loaded in to an output file

getPatternsAsDataFrame()

Complete set of periodic-frequent patterns will be loaded in to a dataframe

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function

getRuntime()

Total amount of runtime taken by the mining process will be retrieved from this function

creatingOneItemSets()

Scan the database and store the items with their timestamps which are periodic frequent

getPeriodAndSupport()

Calculates the support and period for a list of timestamps.

Generation()

Used to implement prefix class equivalence method to generate the periodic patterns recursively

Methods to execute code on terminal

Format:

(.venv) $ python3 PFPMC.py <inputFile> <outputFile> <minSup> <maxPer>

Example usage:

(.venv) $ python3 PFPMC.py sampleDB.txt patterns.txt 10.0 4.0


        .. note:: minSup and maxPer will be considered in percentage of database transactions

Importing this algorithm into a python program

from PAMI.periodicFrequentPattern.basic import PFPMC as alg

obj = alg.PFPMC("../basic/sampleTDB.txt", "2", "5")

obj.mine()

periodicFrequentPatterns = obj.getPatterns()

print("Total number of Periodic Frequent Patterns:", len(periodicFrequentPatterns))

obj.save("patterns")

Df = obj.getPatternsAsDataFrame()

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits:

The complete program was written by P.Likhitha under the supervision of Professor Rage Uday Kiran.

getMemoryRSS() → float[source]

Total amount of RSS memory consumed by the mining process will be retrieved from this function

Returns:

returning RSS memory consumed by the mining process

Return type:

float

getMemoryUSS() → float[source]

Total amount of USS memory consumed by the mining process will be retrieved from this function

Returns:

returning USS memory consumed by the mining process

Return type:

float

getPatterns() → dict[source]

Function to send the set of periodic-frequent patterns after completion of the mining process

Returns:

returning periodic-frequent patterns

Return type:

dict

getPatternsAsDataFrame() → DataFrame[source]

Storing final periodic-frequent patterns in a dataframe

Returns:

returning periodic-frequent patterns in a dataframe

Return type:

pd.DataFrame

getRuntime() → float[source]

Calculating the total amount of runtime taken by the mining process

Returns:

returning total amount of runtime taken by the mining process

Return type:

float

mine() → None[source]

Mining process will start from this function :return: None

printResults() → None[source]

This function is used to print the results :return: None

save(outFile: str) → None[source]

Complete set of periodic-frequent patterns will be loaded in to an output file

Parameters:

outFile (csv file) – name of the output file

Returns:

None

startMine() → None[source]

Mining process will start from this function :return: None

PAMI.periodicFrequentPattern.basic.PSGrowth module

class PAMI.periodicFrequentPattern.basic.PSGrowth.Node(item, children)[source]

Bases: object

A class used to represent the node of frequentPatternTree

Attributes:
itemint

storing item of a node

timeStampslist

To maintain the timeStamps of Database at the end of the branch

parentnode

To maintain the parent of every node

childrenlist

To maintain the children of node

Methods:
addChild(itemName)

storing the children to their respective parent nodes

addChild(node) → None[source]

Appends the children node details to a parent node

Parameters:

node – children node

Returns:

appending children node to parent node

class PAMI.periodicFrequentPattern.basic.PSGrowth.PSGrowth(iFile, minSup, maxPer, sep='\t')[source]

Bases: _periodicFrequentPatterns

Description:

PS-Growth is one of the fundamental algorithm to discover periodic-frequent patterns in a temporal database.

:ReferenceA. Anirudh, R. U. Kiran, P. K. Reddy and M. Kitsuregaway, “Memory efficient mining of periodic-frequent

patterns in transactional databases,” 2016 IEEE Symposium Series on Computational Intelligence (SSCI), 2016, pp. 1-8, https://doi.org/10.1109/SSCI.2016.7849926

Parameters:

  • iFile – str : Name of the Input file to mine complete set of periodic frequent pattern’s

  • oFile – str : Name of the output file to store complete set of periodic frequent pattern’s

  • minSup – str: Controls the minimum number of transactions in which every item must appear in a database.

  • maxPer – str: Controls the maximum number of transactions in which any two items within a pattern can reappear.

  • sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:
iFilefile

Name of the Input file or path of the input file

oFilefile

Name of the output file or path of the output file

minSup: int or float or str

The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float. Example: minSup=10 will be treated as integer, while minSup=10.0 will be treated as float

maxPer: int or float or str

The user can specify maxPer either in count or proportion of database size. If the program detects the data type of maxPer is integer, then it treats maxPer is expressed in count. Otherwise, it will be treated as float. Example: maxPer=10 will be treated as integer, while maxPer=10.0 will be treated as float

sepstr

This variable is used to distinguish items from one another in a transaction. The default separator is tab space or . However, the users can override their default separator.

memoryUSSfloat

To store the total amount of USS memory consumed by the program

memoryRSSfloat

To store the total amount of RSS memory consumed by the program

startTime:float

To record the start time of the mining process

endTime:float

To record the completion time of the mining process

Databaselist

To store the transactions of a database in list

mapSupportDictionary

To maintain the information of item and their frequency

lnoint

it represents the total no of transaction

treeclass

it represents the Tree class

itemSetCountint

it represents the total no of patterns

finalPatternsdict

it represents to store the patterns

Methods:
mine()

Mining process will start from here

getPatterns()

Complete set of patterns will be retrieved with this function

save(oFile)

Complete set of periodic-frequent patterns will be loaded in to an output file

getConditionalPatternsInDataFrame()

Complete set of periodic-frequent patterns will be loaded in to a dataframe

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function

getRuntime()

Total amount of runtime taken by the mining process will be retrieved from this function

OneLengthItems()

Scans the dataset or dataframes and stores in list format

buildTree()

after updating the Databases ar added into the tree by setting root node as null

Methods to execute code on terminal

Format:

(.venv) $ python3 PSGrowth.py <inputFile> <outputFile> <minSup> <maxPer>

Example:

(.venv) $ python3 PSGrowth.py sampleTDB.txt patterns.txt 0.3 0.4



        .. note:: minSup will be considered in percentage of database transactions

Importing this algorithm into a python program

from PAMI.periodicFrequentPattern.basic import PSGrowth as alg

obj = alg.PSGrowth("../basic/sampleTDB.txt", "2", "6")

obj.mine()

periodicFrequentPatterns = obj.getPatterns()

print("Total number of  Patterns:", len(periodicFrequentPatterns))

obj.save("patterns")

Df = obj.getPatternsAsDataFrame()

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits:

The complete program was written by P.Likhitha under the supervision of Professor Rage Uday Kiran.

getMemoryRSS() → float[source]

Total amount of RSS memory consumed by the mining process will be retrieved from this function

Returns:

returning RSS memory consumed by the mining process

Return type:

float

getMemoryUSS() → float[source]

Total amount of USS memory consumed by the mining process will be retrieved from this function

Returns:

returning USS memory consumed by the mining process

Return type:

float

getPatterns() → dict[source]

Function to send the set of periodic-frequent patterns after completion of the mining process

Returns:

returning periodic-frequent patterns

Return type:

dict

getPatternsAsDataFrame() → DataFrame[source]

Storing final periodic-frequent patterns in a dataframe

Returns:

returning periodic-frequent patterns in a dataframe

Return type:

pd.DataFrame

getRuntime() → float[source]

Calculating the total amount of runtime taken by the mining process

Returns:

returning total amount of runtime taken by the mining process

Return type:

float

mine() → None[source]

Mining process will start from this function :return: None

printResults() → None[source]

This function is used to print the results :return: None

save(outFile: str) → None[source]

Complete set of periodic-frequent patterns will be loaded in to an output file

Parameters:

outFile (csv file) – name of the output file

Returns:

None

startMine() → None[source]

Mining process will start from this function :return: None

PAMI.periodicFrequentPattern.basic.PSGrowth.conditionalTransactions(patterns, timestamp) → Tuple[List[List[int]], List[List[_Interval]], Dict[int, Tuple[int, int]]][source]

To sort and update the conditional transactions by removing the items which fails frequency and periodicity conditions

Parameters:

  • patterns – conditional patterns of a node

  • timestamp – timeStamps of a conditional pattern

Returns:

conditional transactions with their respective timeStamps

PAMI.periodicFrequentPattern.basic.PSGrowth.getPeriodAndSupport(timeStamps) → List[int][source]

Calculates the period and support of list of timeStamps

Parameters:

timeStamps – timeStamps of a pattern or item

Returns:

support and periodicity

PAMI.periodicFrequentPattern.basic.abstract module

PAMI.periodicFrequentPattern.basic.parallelPFPGrowth module

class PAMI.periodicFrequentPattern.basic.parallelPFPGrowth.Node(item, count, children)[source]

Bases: object

A class used to represent the node of frequentPatternTree

Attributes:
itemint or None

Storing item of a node

timeStampslist

To maintain the timestamps of a database at the end of the branch

parentnode

To maintain the parent of every node

childrenlist

To maintain the children of a node

countint

To maintain the count of every node

Methods:
addChild(itemName)

Storing the children to their respective parent nodes

toString()

To print the node

addChild(node)[source]

To add the children to a node

Parameters:

node – children of a node

toString(level=0)[source]

To print the node

Parameters:

level – level of a node

class PAMI.periodicFrequentPattern.basic.parallelPFPGrowth.PFPTree[source]

Bases: object

A class used to represent the periodic frequent pattern tree

Attributes:
rootnode

To maintain the root of the tree

summariesdict

To maintain the summary of the tree

Methods:
add(basket, tid, count)

To add the basket to the tree

getTransactions()

To get the transactions of the tree

merge(tree)

To merge the tree

project(itemId)

To project the tree

satisfyPer(tids, maxPer, numTrans)

To satisfy the periodicity constraint

extract(minCount, maxPer, numTrans, isResponsible = lambda x:True)

To extract the periodic frequent patterns

add(basket, tid, count)[source]

To add the basket to the tree

Parameters:

  • basket – basket of a database

  • tid – timestamp of a database

  • count – count of a node

extract(minCount, maxPer, numTrans, isResponsible=<function PFPTree.<lambda>>)[source]

To extract the periodic frequent patterns

Parameters:

  • minCount – minimum count of a node

  • maxPer – maximum periodicity

  • numTrans – number of transactions

  • isResponsible – responsible node of a tree

getTransactions()[source]

To get the transactions of the tree

Returns:

returning the transactions of the tree

merge(tree)[source]

To merge the tree

Parameters:

tree – tree of a database

project(itemId)[source]

To project the tree

Parameters:

itemId – item of a node

satisfyPer(tids, maxPer, numTrans)[source]

To satisfy the periodicity constraint

Parameters:

  • tids – timestamps of a database

  • maxPer – maximum periodicity

  • numTrans – number of transactions

class PAMI.periodicFrequentPattern.basic.parallelPFPGrowth.Summary(count, nodes)[source]

Bases: object

A class used to represent the summary of the tree

Attributes:
countint

To maintain the count of a node

nodeslist

To maintain the nodes of a tree

tidsset

To maintain the timestamps of a database

class PAMI.periodicFrequentPattern.basic.parallelPFPGrowth.parallelPFPGrowth(iFile, minSup, maxPer, numWorker)[source]

Bases: _periodicFrequentPatterns

Description:

ParallelPFPGrowth is one of the fundamental distributed algorithm to discover periodic-frequent patterns in a transactional database. It is based PySpark framework.

Reference:

  1. Saideep, R. Uday Kiran, Koji Zettsu, Cheng-Wei Wu, P. Krishna Reddy, Masashi Toyoda, Masaru Kitsuregawa: Parallel Mining of Partial Periodic Itemsets in Big Data. IEA/AIE 2020: 807-819

Parameters:

  • iFile – str : Name of the Input file to mine complete set of periodic frequent pattern’s

  • oFile – str : Name of the output file to store complete set of periodic frequent pattern’s

  • minSup – str: Controls the minimum number of transactions in which every item must appear in a database.

  • maxPer – str: Controls the maximum number of transactions in which any two items within a pattern can reappear.

  • sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:
iFilefile

Name of the Input file or path of the input file

oFilefile

Name of the output file or path of the output file

minSupint or float or str

The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float. Example: minSup=10 will be treated as integer, while minSup=10.0 will be treated as float

maxPerint or float or str

The user can specify maxPer either in count or proportion of database size. If the program detects the data type of maxPer is integer, then it treats maxPer is expressed in count. Otherwise, it will be treated as float. Example: maxPer=10 will be treated as integer, while maxPer=10.0 will be treated as float

numWorkerint

The user can specify the number of worker machines to be employed for finding periodic-frequent patterns.

sepstr

This variable is used to distinguish items from one another in a transaction. The default seperator is tab space or . However, the users can override their default separator.

memoryUSSfloat

To store the total amount of USS memory consumed by the program

memoryRSSfloat

To store the total amount of RSS memory consumed by the program

startTimefloat

To record the start time of the mining process

endTimefloat

To record the completion time of the mining process

Databaselist

To store the transactions of a database in list

mapSupportDictionary

To maintain the information of item and their frequency

lnoint

To represent the total no of transaction

treeclass

To represents the Tree class

itemSetCountint

To represents the total no of patterns

finalPatternsdict

To store the complete patterns

Methods:
mine()

Mining process will start from here

getPatterns()

Complete set of patterns will be retrieved with this function

save(oFile)

Complete set of periodic-frequent patterns will be loaded in to a output file

getPatternsAsDataFrame()

Complete set of periodic-frequent patterns will be loaded in to a dataframe

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function

getRuntime()

Total amount of runtime taken by the mining process will be retrieved from this function

creatingItemSets(fileName)

Scans the dataset and stores in a list format

PeriodicFrequentOneItem()

Extracts the one-periodic-frequent patterns from database

updateDatabases()

Update the database by removing aperiodic items and sort the Database by item decreased support

buildTree()

After updating the Database, remaining items will be added into the tree by setting root node as null

convert()

to convert the user specified value

Methods to execute code on terminal

Format:

(.venv) $ python3 parallelPFPGrowth.py <inputFile> <outputFile> <minSup> <maxPer> <noWorker>

Example usage:

(.venv) $ python3 parallelPFPGrowth.py sampleTDB.txt patterns.txt 0.3 0.4 5


        .. note:: minSup will be considered in percentage of database transactions

Importing this algorithm into a python program

from PAMI.periodicFrequentPattern.basic import parallelPFPGrowth as alg

obj = alg.parallelPFPGrowth(iFile, minSup, maxPer, numWorkers, sep='    ')

obj.mine()

periodicFrequentPatterns = obj.getPatterns()

print("Total number of Periodic Frequent Patterns:", len(periodicFrequentPatterns))

obj.save(oFile)

Df = obj.getPatternsAsDataFrame()

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)
Mine()[source]

Start the mining process

func1(ps1, tid)[source]

Add the tid to the set

Parameters:

  • ps1 – set

  • tid – timestamp of a database

return: set

func2(ps1, ps2)[source]

Union of two sets

Parameters:

  • ps1 – set.

  • ps2 – set

return: set

func3(tids, endts)[source]

Calculate the periodicity of a transaction

Parameters:

tids – timestamps of a database

return: periodicity :param endts: last timestamp return: periodicity

genCondTransactions(tid, basket, rank, nPartitions)[source]

Get the conditional transactions from the database

Parameters:

  • tid – timestamp of a database

  • basket – basket of a database

  • rank – rank of a database

  • nPartitions – number of partitions

getFrequentItems(data)[source]

Get the frequent items from the database

Parameters:

data – database

Returns:

frequent items

getFrequentItemsets(data, freqItems)[source]

Get the frequent itemsets from the database

Parameters:

  • data – database

  • freqItems – frequent items

Returns:

frequent itemsets

getMemoryRSS()[source]

Total amount of RSS memory consumed by the mining process will be retrieved from this function

Returns:

returning RSS memory consumed by the mining process

Return type:

float

getMemoryUSS()[source]

Total amount of USS memory consumed by the mining process will be retrieved from this function

Returns:

returning USS memory consumed by the mining process

Return type:

float

getPartitionId(key, nPartitions)[source]

Get the partition id

Parameters:

  • key – key of a database

  • nPartitions – number of partitions.

Returns:

partition id

getPatterns()[source]

Function to send the set of frequent patterns after completion of the mining process

Returns:

returning frequent patterns

Return type:

dict

getPatternsAsDataFrame()[source]

Storing final frequent patterns in a dataframe

Returns:

returning frequent patterns in a dataframe

Return type:

pd.DataFrame

getRuntime()[source]

Calculating the total amount of runtime taken by the mining process

Returns:

returning total amount of runtime taken by the mining process

Return type:

float

printResults()[source]

This function is used to print the results

save(outFile)[source]

Complete set of frequent patterns will be loaded in to a output file

Parameters:

outFile (file) – name of the output file

startMine()[source]

Start the mining process

Module contents