a ɝh1@sddlmZmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z ddlmZmZddlmZddlmZeddddd d d fd d Zd dZGdddeZGdddeZGdddeZGdddeZdS))print_functiondivisionN) Structure)Dataset DataLoader)default_collate)SubsetRandomSampler@g?Fc Ksbt|} | ddur^|durJ||dks,Jd||}td|dn|||dks^Jtt| } | dr|| d} n t|| } | dr| d} n t|| } | dr| d}n t|| }t| d| }t| ||  | }|rt| | d}t||||||d}t||||||d}|rFt||||||d}|rV|||fS||fSdS) a Utility function for dividing a dataset to train, val, test datasets. !!! The dataset needs to be shuffled before using the function !!! Parameters ---------- dataset: torch.utils.data.Dataset The full dataset to be divided. collate_fn: torch.utils.data.DataLoader batch_size: int train_ratio: float val_ratio: float test_ratio: float return_test: bool Whether to return the test dataset loader. If False, the last test_size data will be hidden. num_workers: int pin_memory: bool Returns ------- train_loader: torch.utils.data.DataLoader DataLoader that random samples the training data. val_loader: torch.utils.data.DataLoader DataLoader that random samples the validation data. (test_loader): torch.utils.data.DataLoader DataLoader that random samples the test data, returns if return_test=True. train_sizeNr zB[Warning] train_ratio is None, using 1 - val_ratio - test_ratio = z as training data. test_sizeZval_size) batch_sizesampler num_workers collate_fn pin_memory)lenprintlistrangeintrr)datasetrr Z train_ratioZ val_ratioZ test_ratioZ return_testrrkwargs total_sizeindicesr r Z valid_sizeZ train_samplerZ val_samplerZ test_samplerZ train_loader val_loader test_loaderr$/var/www/html/AI4Kappa/cgcnn/data.pyget_train_val_test_loadersX"           rcCsggg}}}gg}}g}d}t|D]z\}\\} } } } } | jd}|| || || |tt||}|||| || ||7}q*tj|ddtj|ddtj|dd|ftj|dd|fS)a Collate a list of data and return a batch for predicting crystal properties. Parameters ---------- dataset_list: list of tuples for each data point. (atom_fea, nbr_fea, nbr_fea_idx, target) atom_fea: torch.Tensor shape (n_i, atom_fea_len) nbr_fea: torch.Tensor shape (n_i, M, nbr_fea_len) nbr_fea_idx: torch.LongTensor shape (n_i, M) target: torch.Tensor shape (1, ) cif_id: str or int Returns ------- N = sum(n_i); N0 = sum(i) batch_atom_fea: torch.Tensor shape (N, orig_atom_fea_len) Atom features from atom type batch_nbr_fea: torch.Tensor shape (N, M, nbr_fea_len) Bond features of each atom's M neighbors batch_nbr_fea_idx: torch.LongTensor shape (N, M) Indices of M neighbors of each atom crystal_atom_idx: list of torch.LongTensor of length N0 Mapping from the crystal idx to atom idx target: torch.Tensor shape (N, 1) Target value for prediction batch_cif_ids: list r)dim) enumerateshapeappendtorch LongTensornparangecatstack)Z dataset_listZbatch_atom_feaZ batch_nbr_feaZbatch_nbr_fea_idxZcrystal_atom_idxZ batch_target batch_cif_idsZbase_idxiatom_feanbr_fea nbr_fea_idxtargetcif_idn_inew_idxrrr collate_poolbs.!            r3c@s"eZdZdZdddZddZdS)GaussianDistancezE Expands the distance by Gaussian basis. Unit: angstrom NcCsF||ks J|||ksJt|||||_|dur<|}||_dS)z Parameters ---------- dmin: float Minimum interatomic distance dmax: float Maximum interatomic distance step: float Step size for the Gaussian filter N)r&r'filtervar)selfdmindmaxstepr6rrr__init__s zGaussianDistance.__init__cCs*t|dtjf|jd |jdS)aw Apply Gaussian disntance filter to a numpy distance array Parameters ---------- distance: np.array shape n-d array A distance matrix of any shape Returns ------- expanded_distance: shape (n+1)-d array Expanded distance matrix with the last dimension of length len(self.filter) .)r&expnewaxisr5r6)r7 distancesrrrexpandszGaussianDistance.expand)N)__name__ __module__ __qualname____doc__r;r@rrrrr4s r4c@s8eZdZdZddZddZddZdd Zd d Zd S) AtomInitializerzz Base class for intializing the vector representation for atoms. !!! Use one AtomInitializer per dataset !!! cCst||_i|_dSN)set atom_types _embedding)r7rHrrrr;s zAtomInitializer.__init__cCs||jvsJ|j|SrF)rHrI)r7 atom_typerrr get_atom_feaszAtomInitializer.get_atom_feacCs0||_t|j|_dd|jD|_dS)NcSsi|]\}}||qSrr.0rJidxrrr z3AtomInitializer.load_state_dict..)rIrGkeysrHitems _decodedict)r7 state_dictrrrload_state_dicts zAtomInitializer.load_state_dictcCs|jSrF)rIr7rrrrTszAtomInitializer.state_dictcCs*t|ds dd|jD|_|j|S)NrScSsi|]\}}||qSrrrLrrrrOrPz*AtomInitializer.decode..)hasattrrIrRrS)r7rNrrrdecodes  zAtomInitializer.decodeN) rArBrCrDr;rKrUrTrXrrrrrEs rEcs eZdZdZfddZZS)AtomCustomJSONInitializera Initialize atom feature vectors using a JSON file, which is a python dictionary mapping from element number to a list representing the feature vector of the element. Parameters ---------- elem_embedding_file: str The path to the .json file cst|}t|}Wdn1s(0Ydd|D}t|}tt|||D]\}}t j |t d|j |<qhdS)NcSsi|]\}}t||qSr)r)rMkeyvaluerrrrOrPz6AtomCustomJSONInitializer.__init__..)dtype) openjsonloadrRrGrQsuperrYr;r&arrayfloatrI)r7Zelem_embedding_filefZelem_embeddingrHrZr[ __class__rrr;s ( z"AtomCustomJSONInitializer.__init__)rArBrCrDr; __classcell__rrrdrrYs rYc@s*eZdZdZdddZd d Zd d Zd S)CIFDatau The CIFData dataset is a wrapper for a dataset where the crystal structures are stored in the form of CIF files. The dataset should have the following directory structure: root_dir ├── id_prop.csv ├── atom_init.json ├── id0.cif ├── id1.cif ├── ... id_prop.csv: CSV file that contains two columns. The first column recodes a unique ID for each crystal, and the second column recodes the value of target property. atom_init.json: JSON file that contains the initialization vector for each element. ID.cif: CIF file that recodes the crystal structure, where ID is the unique ID for the crystal. Parameters ---------- root_dir: str The path to the root directory of the dataset max_num_nbr: int The maximum number of neighbors while constructing the crystal graph radius: float The cutoff radius for searching neighbors dmin: float The minimum distance for constructing GaussianDistance step: float The step size for constructing GaussianDistance random_seed: int Random seed for shuffling the dataset Returns ------- atom_fea: torch.Tensor shape (n_i, atom_fea_len) nbr_fea: torch.Tensor shape (n_i, M, nbr_fea_len) nbr_fea_idx: torch.LongTensor shape (n_i, M) target: torch.Tensor shape (1, ) cif_id: str or int r皙?{c Cs||_|||_|_tj|s(Jdtj|jd}tj|sLJdt|2}t |} t | dd| D|_ Wdn1s0Yt |t |j tj|jd} tj| sJdt| |_t||j|d|_dS) Nzroot_dir does not exist!z id_prop.csvzid_prop.csv does not exist!cSsg|]}|qSrr)rMrowrrr 6rPz$CIFData.__init__..zatom_init.jsonzatom_init.json does not exist!)r8r9r:)root_dir max_num_nbrradiusospathexistsjoinr]csvreadernext id_prop_datarandomseedshufflerYarir4gdf) r7rnrorpr8r: random_seedZ id_prop_filercrvZatom_init_filerrrr;,s  .   zCIFData.__init__cCs t|jSrF)rrxrVrrr__len__>szCIFData.__len__c sj|\}}ttjj|tfddt t D}t |}j jdd}dd|D}gg}}|D]}t |jkrtd||ttdd|d gjt ||ttd d|jd gjt |q~|ttd d|dj|ttd d|djq~t|t|}}j|}t |}t |}t |}t t|g}|||f||fS)Ncs g|]}j|jjqSr)r|rKspecienumber)rMr+Zcrystalr7rrrmEsz'CIFData.__getitem__..T) include_indexcSsg|]}t|dddqS)cSs|dSNr rxrrrIrPz0CIFData.__getitem__...)rZ)sorted)rMZnbrsrrrrmIrPz`{} not find enough neighbors to build graph. If it happens frequently, consider increase radius.cSs|dSNr<rrrrrrPrPz%CIFData.__getitem__..rcSs|dSrrrrrrrRrPg?cSs|dSrrrrrrrVrPcSs|dSrrrrrrrXrP)rxr from_filerqrrrtrnr&vstackrrr$Tensorget_all_neighborsrprowarningswarnformatr#rmaprar}r@r%rb) r7rNr0r/r,Zall_nbrsr.r-nbrrrr __getitem__AsJ         zCIFData.__getitem__N)rhrirrjrk)rArBrCrDr;rrrrrrrgs / rg) __future__rrru functoolsr^rqryrnumpyr&r$pymatgen.core.structurertorch.utils.datarrtorch.utils.data.dataloaderrtorch.utils.data.samplerrrr3objectr4rErYrgrrrrs,    P8-