A Temporal Extension Library for PyTorch Geometric


PyPI Version Docs Status Repo size Code Coverage Build Status

Documentation | External Resources | Datasets

PyTorch Geometric Temporal is a temporal (dynamic) extension library for PyTorch Geometric.

The library consists of various dynamic and temporal geometric deep learning, embedding, and spatio-temporal regression methods from a variety of published research papers. In addition, it consists of an easy-to-use dataset loader and iterator for dynamic and temporal graphs, gpu-support. It also comes with a number of benchmark datasets with temporal and dynamic graphs (you can also create your own datasets).


Citing

If you find PyTorch Geometric Temporal and the new datasets useful in your research, please consider adding the following citation:

@misc{pytorch_geometric_temporal,
      author = {Benedek, Rozemberczki and Paul, Scherer},
      title = {{PyTorch Geometric Temporal}},
      year = {2020},
      publisher = {GitHub},
      journal = {GitHub repository},
      howpublished = {\url{https://github.com/benedekrozemberczki/pytorch_geometric_temporal}},
}

A simple example

PyTorch Geometric Temporal makes implementing Dynamic and Temporal Graph Neural Networks quite easy - see the accompanying tutorial. For example, this is all it takes to implement a recurrent graph convolutional network with two consecutive graph convolutional GRU cells and a linear layer:

import torch
import torch.nn.functional as F
from torch_geometric_temporal.nn.recurrent import GConvGRU

class RecurrentGCN(torch.nn.Module):

    def __init__(self, node_features, num_classes):
        super(RecurrentGCN, self).__init__()
        self.recurrent_1 = GConvGRU(node_features, 32, 5)
        self.recurrent_2 = GConvGRU(32, 16, 5)
        self.linear = torch.nn.Linear(16, num_classes)

    def forward(self, x, edge_index, edge_weight):
        x = self.recurrent_1(x, edge_index, edge_weight)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.recurrent_2(x, edge_index, edge_weight)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.linear(x)
        return F.log_softmax(x, dim=1)

Methods Included

In detail, the following temporal graph neural networks were implemented.

Discrete Recurrent Graph Convolutions

Temporal Graph Convolutions

Auxiliary Graph Convolutions


Head over to our documentation to find out more about installation, creation of datasets and a full list of implemented methods and available datasets. For a quick start, check out the examples in the examples/ directory.

If you notice anything unexpected, please open an issue. If you are missing a specific method, feel free to open a feature request.


Installation

PyTorch 1.7.0

To install the binaries for PyTorch 1.7.0, simply run

$ pip install torch-scatter==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.7.0.html
$ pip install torch-sparse==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.7.0.html
$ pip install torch-cluster==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.7.0.html
$ pip install torch-spline-conv==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.7.0.html
$ pip install torch-geometric
$ pip install torch-geometric-temporal

where ${CUDA} should be replaced by either cpu, cu92, cu101, cu102 or cu110 depending on your PyTorch installation.

cpu cu92 cu101 cu102 cu110
Linux
Windows
macOS

PyTorch 1.6.0

To install the binaries for PyTorch 1.6.0, simply run

$ pip install torch-scatter==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.6.0.html
$ pip install torch-sparse==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.6.0.html
$ pip install torch-cluster==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.6.0.html
$ pip install torch-spline-conv==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.6.0.html
$ pip install torch-geometric
$ pip install torch-geometric-temporal

where ${CUDA} should be replaced by either cpu, cu92, cu101 or cu102 depending on your PyTorch installation.

cpu cu92 cu101 cu102
Linux
Windows
macOS

Running tests

$ python setup.py test

Running examples

$ cd examples
$ python gconvgru_example.py

License

Owner
Benedek Rozemberczki
PhD candidate at The University of Edinburgh @cdt-data-science working on machine learning and data mining related to graph structured data.
Benedek Rozemberczki
Comments
  • temporal graph classification

    temporal graph classification

    I'm wondering if there is an example of classification carried out for data represented as temporal graphs?

    To be precise: in my use case I have for instance a sequence of 100 graphs representing together a single entity/object. Next, I have 200 entities. Is there an easy way to input those 200 data instances (with ground truth classification labels) to one of the numerous deep learning architectures implemented in this repo to obtain instance representations useful in the classification task? Or simply obtain predicted labels? Is there an example for this or did I miss it ? (apologies if I did).

    Thank you for your help in advance.

  • Question about example code

    Question about example code

    Hi, I've been trying to reuse the example code, MPNNLSTM, GCLSTM, and GConvLSTM with my own dataset. My MSE errors are around .2 but the models do not return correct target values. When I run the examples the MSE is around 1. What does MSE represent? How should I structure data my data for these models? I have a graph with 20 nodes, 2 features per node, and 32 edges. The dataset has 360 separate graphs. My dataset looks similar to the one used in MPNNLSTM.

  • [Bug?] the way to set GCN.weight in EvolveGCN.

    [Bug?] the way to set GCN.weight in EvolveGCN.

    Hi @benedekrozemberczki, I found that the gradient function may be lost in the implementation of EvolveGCN. The reason for this should be the following line of code: https://github.com/benedekrozemberczki/pytorch_geometric_temporal/blob/bd6fac946378507a729b5723e9ace46a9dc0cbe1/torch_geometric_temporal/nn/recurrent/evolvegcno.py#L69 And here is a demo try to verify it:

    import torch
    
    torch.manual_seed(123)
    
    
    class MyModel(torch.nn.Module):
        def __init__(self):
            super().__init__()
            self.toy_model = ToyModel(weight=True)
    
        def forward(self, x):
            W = self.toy_model.weight
            self.toy_model.weight = torch.nn.parameter.Parameter(W)
            return self.toy_model(x)
    
    
    class MyModelParam(torch.nn.Module):
        def __init__(self):
            super().__init__()
            self.W = torch.nn.parameter.Parameter(torch.Tensor(2, 3).fill_(0.5))
            self.toy_model = ToyModel(weight=False)
    
        def forward(self, x):
            W = self.W
            return self.toy_model(x, weight=W)
    
    
    class ToyModel(torch.nn.Module):
        def __init__(self, weight=True):
            super().__init__()
            if weight:
                self.weight = torch.nn.parameter.Parameter(torch.Tensor(2, 3).fill_(0.5))
            else:
                self.register_parameter('weight', None)
            # self.func = torch.nn.parameter.Parameter(torch.Tensor(2, 2).fill_(0.1))
    
        def forward(self, x, weight=None):
            if weight is not None:
                w = weight
            else:
                w = self.weight
            # x = self.func * x
            x = torch.matmul(x, w)
            return x
    
    
    x = torch.Tensor(1, 2).fill_(0.8)
    y = torch.Tensor(1, 3).fill_(0.7)
    print("################## Test 1: reset GCN.weight by nn.Parameter()##################")
    model = MyModel()
    print("toyModel weight before:")
    print(model.toy_model.weight)
    optim = torch.optim.SGD(model.parameters(), lr=1e-2)
    
    # print("x: {}".format(x))
    # print("y: {}".format(y))
    prediction = model(x)
    loss = (prediction - y).sum()
    loss.backward()
    optim.step()
    print("toyModel weight after:")
    print(model.toy_model.weight)
    
    ########################## Test 2
    print("################## Test 2: pass weight as a parameter during forward##################")
    model_param = MyModelParam()
    print("model param weight before:")
    print(model_param.W)
    optim_param = torch.optim.SGD(model_param.parameters(), lr=1e-2)
    x_param = torch.Tensor(1, 2).fill_(0.8)
    y_param = torch.Tensor(1, 3).fill_(0.7)
    # print("x_param: {}".format(x_param))
    # print("y_param: {}".format(y_param))
    prediction_param = model_param(x_param)
    loss_param = (prediction_param - y_param).sum()
    loss_param.backward()
    optim_param.step()
    print("model param weight after:")
    print(model_param.W)
    

    result is here:

    ################## Test 1: reset GCN.weight by nn.Parameter()##################
    toyModel weight before:
    Parameter containing:
    tensor([[0.5000, 0.5000, 0.5000],
            [0.5000, 0.5000, 0.5000]], requires_grad=True)
    toyModel weight after:
    Parameter containing:
    tensor([[0.5000, 0.5000, 0.5000],
            [0.5000, 0.5000, 0.5000]], requires_grad=True)
    ################## Test 2: pass weight as a parameter during forward##################
    model param weight before:
    Parameter containing:
    tensor([[0.5000, 0.5000, 0.5000],
            [0.5000, 0.5000, 0.5000]], requires_grad=True)
    model param weight after:
    Parameter containing:
    tensor([[0.4920, 0.4920, 0.4920],
            [0.4920, 0.4920, 0.4920]], requires_grad=True)
    
    Process finished with exit code 0
    

    We can see that if we set model.weight in each batch iter by nn.paramter.Parameter(), the weight has not been updated.
    One way to fix it should be to pass a new weight during every forward. But pyG does not seem to provide the corresponding interface.

  • Tutorial for creating custom dataset for multiple dynamic graphs?

    Tutorial for creating custom dataset for multiple dynamic graphs?

    Hi Benedek!

    I wonder if there is a tutorial for creating a custom dataset for multiple dynamic graphs?

    I want to follow a code snippet so I can play around to learn how to do that.

    Thanks in advance. :)

  • First commit for heterogeneous graph support

    First commit for heterogeneous graph support

    This includes:

    • a new data-structure StaticHeteroGraphTemporalSignal that works like StaticGraphTemporalSignal but with torch_geometric HeteroData objects as snapshots; instead of np arrays, dictionaries wirth key(node/edge types as strings)/value(indices/features as np arrays) are expected
    • code tests for the new data-structure
    • documentation for the new data-structure

    Feedback is welcome! If you think it makes sense to extend your package like that it would be great to open a new branch for heterogeneous support where we can go on working in this direction. Thanks and cheers, Gregor

  • Extreme RAM consumption by multilayer models

    Extreme RAM consumption by multilayer models

    Hi Benedek! First of all, thank you for this project, I hope it has a bright future ahead. My issue is with building deep/multi-layer models out of Recurrent Graph Convolutional Layers. There are no examples of it in the repo (or anywhere else on the internet as far as I searched), so I might be doing something wrong. Here is my project. Now, what I observe when running this code is that RAM utilization goes up nearly exponentially with the number of RGC layers. Of course, most of it ends up in swap, making the training process too slow to be viable. This does not appear to be a memory leak, since most of the memory is not used by Python objects, but rather internally by PyTorch. Have you encountered this issue and is there a way to fix it?

  • How to distribute the model to compute in multiple GPUs?

    How to distribute the model to compute in multiple GPUs?

    I am trying to distribute the model computation in multiple GPUs using DataParallel from Pytorch Geometric library I was trying to follow this example but I am running into errors. Is this the way to do it or should I look somewhere else? Are there any examples out there to distribute models from Pytorch Geometric Temporal library?

  • Combining multiple StaticGraphTemporalSignal objects into StaticGraphTemporalSignalBatch objects

    Combining multiple StaticGraphTemporalSignal objects into StaticGraphTemporalSignalBatch objects

    Hi,

    I've created a Dataset object which currently returns a single sequence (StaticGraphTemporalSignal) from a directory of many sequences. When feeding this to a DataLoader, I receive lists of length batch_size instead of batched objects. Is there a straightforward procedure of combining multiple StaticGraphTemporalSignal objects into a StaticGraphTemporalSignalBatch object? I have not found any examples of StaticGraphTemporalSignalBatch in use.

    Cheers.

  • Putting multiple graphs in one StaticGraphTemporalSignal iterator

    Putting multiple graphs in one StaticGraphTemporalSignal iterator

    I'm dealing with a basketball players trajectory forecasting dataset; it is both temporal and spatial with the shape: [number of plays, number of timesteps, player, position]. Each play is composed of several timesteps, so in total I would have number of plays * number of timesteps graphs.

    From reading documentation and source code it seems that the StaticGraphTemporalSignal accepts multiple graphs, but I'm not sure how to do that. Moreover, I cannot create an instance of this iterator without passing target labels, which is not what I aim for in this project; unless I pass the target labels as the features of the next step in the prediction.

    I'm not sure if my question is clear I would be glad to elaborate, I require help :)

  • Not all tests pass on GPU environment

    Not all tests pass on GPU environment

    Hi all, just thought I would let you know that not all your tests pass when running on a machine with a GPU

    I was looking at the MTGNN test which started working once I set the device to CPU and wasn't working with the existing code

    
       #device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
        device = torch.device("cpu")
    

    =========================================================================== short test summary info ============================================================================ FAILED test/convolutional_test.py::test_astgcn - RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! FAILED test/convolutional_test.py::test_mstgcn - RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! FAILED test/convolutional_test.py::test_gman - RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _thnn_conv2d_forward =================================================================== 3 failed, 33 passed in 67.57s (0:01:07) ==================================================================== (pytorch_latest_p36) sh-4.2$

  • Feature request for Graph-Structure-Learning GNN for temporal Graph

    Feature request for Graph-Structure-Learning GNN for temporal Graph

    Is there a plan to include Graph-Structure-Learning graph neural network models for temporal graphs? A few models with github repos include MTGNN(https://github.com/nnzhan/MTGNN) and Graph-WaveNet (https://github.com/nnzhan/Graph-WaveNet)

  • Classification of a sequence of graphs

    Classification of a sequence of graphs

    Hello!

    In my project, I work on motion classification and was wondering if pytorch geometric temporal could be used for this application. Namely, my dataset consists of a sequence of graphs such that each graph represents a hand pose, where joints are given as nodes and bones are given as edges. The graph is static with temporal signal (3 dynamic features in each of 21 nodes).

    I implemented a GCN for graph classification using pytorch geometric, which takes the following data representation: Data(x=[21, 3], edge_index=[2, 40], y=[1])

    Now, I would like to use the temporal relationship between the graphs and classify their whole sequence instead of a single graph. However, I encountered the following issues: -StaticGraphTemporalSignal requires to define targets (labels) for each graph, which is in conflict with my case -I could not pass entire sequence of graphs to the neural network, eg. DCRNN. For instance, examples given on the website calculate loss for each graph separately, and not for the sequence of graphs as a whole.

    My approach is quite similar to the work presented in this paper and I was wondering if using pytorch geometric temporal for this case would be possible. Any advice will be appreciated!

    Best, Bartek

  • issue window vs in_channels

    issue window vs in_channels

    Sorry to reopen #174 ... but it doesn't work with X having this dimensions: (n_nodes, n_features, window). If I run your example of MPNNLSTM and switch node features with window, even if I reshape the x, it doesn't work and I get this error:

    [/usr/local/lib/python3.7/dist-packages/torch_geometric_temporal/nn/recurrent/mpnn_lstm.py] in forward(self, X, edge_index, edge_weight) 80 R = list() 81 ---> 82 S = X.view(-1, self.window, self.num_nodes, self.in_channels)

    RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

    I changed the model definition: self.recurrent = MPNNLSTM(1, 32, 32, 20, node_features, 0.5) and the input x in the snapshot: snapshot.x = snapshot.x.reshape(20,1,4)

  • Separate periods in periods_in and periods_out when defining the model TemporalGNN.

    Separate periods in periods_in and periods_out when defining the model TemporalGNN.

    Hi, I've tried to have a prediction window longer then input window and the notebook doesn't work, i.e. num_timesteps_in=12< num_timesteps_out=14. Because in TemporalGNN you use periods for both the window of x and the window of the prediction (y).

    I suggest this:

    ` class TemporalGNN(torch.nn.Module): def init(self, node_features, periods_in,periods_pred, batch_size): super(TemporalGNN, self).init() # Attention Temporal Graph Convolutional Cell self.tgnn = A3TGCN2(in_channels=node_features, out_channels=32, periods=periods_in,batch_size=batch_size) # node_features=2, periods=12 # Equals single-shot prediction self.linear = torch.nn.Linear(32, periods_pred)

    def forward(self, x, edge_index):
        """
        x = Node features for T time steps
        edge_index = Graph edge indices
        """
        h = self.tgnn(x, edge_index) # x [b, 207, 2, 12]  returns h [b, 207, 12]
        h = F.relu(h) 
        h = self.linear(h)
        return h
    

    `

  • Predicting Path Failure via LRGCN

    Predicting Path Failure via LRGCN

    Hey,

    thank you very much for providing this nice framework. I came across Predicting Path Failure In Time-Evolving Graphs and wanted to use it within this framework (as it is cited in the docs).

    However, looking at the example at LRGCN, I'm quite confused about the passed parameters and the dissimilarity to the tensorflow implementation.

    The original implementation calculates the loss using path classes whether they fail or not. In pytorch geometric temporal the model returns only future node values. How do I incorporate the "precited path failure" here? Is it possible at all to make it work?

    Your help would be much appreciated.

    Best, René

    I added the code for quick reference:

    class RecurrentGCN(torch.nn.Module):
        def __init__(self, node_features):
            super(RecurrentGCN, self).__init__()
            self.recurrent = LRGCN(node_features, 32, 1, 1)
            self.linear = torch.nn.Linear(32, 1)
    
        def forward(self, x, edge_index, edge_weight, h_0, c_0):
            h_0, c_0 = self.recurrent(x, edge_index, edge_weight, h_0, c_0)
            h = F.relu(h_0)
            h = self.linear(h)
            return h, h_0, c_0
    
    model = RecurrentGCN(node_features=4)
    optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
    model.train()
    
    for epoch in tqdm(range(200)):
        cost = 0
        h, c = None, None
        for time, snapshot in enumerate(train_dataset):
            y_hat, h, c = model(snapshot.x, snapshot.edge_index, snapshot.edge_attr, h, c)
            cost = cost + torch.mean((y_hat - snapshot.y) ** 2)
        cost = cost / (time + 1)
        cost.backward()
        optimizer.step()
        optimizer.zero_grad()
    
CVPR2021: Temporal Context Aggregation Network for Temporal Action Proposal Refinement
CVPR2021: Temporal Context Aggregation Network for Temporal Action Proposal Refinement

Temporal Context Aggregation Network - Pytorch This repo holds the pytorch-version codes of paper: "Temporal Context Aggregation Network for Temporal

Jun 28, 2022
Implementation of temporal pooling methods studied in [ICIP'20] A Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment

Implementation of temporal pooling methods studied in [ICIP'20] A Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment

May 16, 2022
Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity
Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

This repository is the official PyTorch implementation of Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

Apr 22, 2022
Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021
Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Geometric Vector Perceptron Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biom

Jun 13, 2022
A PyTorch implementation of "DGC-Net: Dense Geometric Correspondence Network"

DGC-Net: Dense Geometric Correspondence Network This is a PyTorch implementation of our work "DGC-Net: Dense Geometric Correspondence Network" TL;DR A

Jun 8, 2022
Pytorch Geometric Tutorials

Pytorch Geometric Tutorials

Jun 27, 2022
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Jun 23, 2022
Code for "FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection", ICRA 2021
Code for

FGR This repository contains the python implementation for paper "FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection"(I

Jun 1, 2022
Geometric Vector Perceptron --- a rotation-equivariant GNN for learning from biomolecular structure

Geometric Vector Perceptron Code to accompany Learning from Protein Structure with Geometric Vector Perceptrons by B Jing, S Eismann, P Suriana, RJL T

Jul 2, 2022
Certifiable Outlier-Robust Geometric Perception

Certifiable Outlier-Robust Geometric Perception About This repository holds the implementation for certifiably solving outlier-robust geometric percep

Jul 6, 2022
A geometric deep learning pipeline for predicting protein interface contacts.
A geometric deep learning pipeline for predicting protein interface contacts.

A geometric deep learning pipeline for predicting protein interface contacts.

Jun 30, 2022
3DMV jointly combines RGB color and geometric information to perform 3D semantic segmentation of RGB-D scans.
3DMV jointly combines RGB color and geometric information to perform 3D semantic segmentation of RGB-D scans.

3DMV 3DMV jointly combines RGB color and geometric information to perform 3D semantic segmentation of RGB-D scans. This work is based on our ECCV'18 p

Feb 6, 2022
Code for "SRHEN: Stepwise-Refining Homography Estimation Network via Parsing Geometric Correspondences in Deep Latent Space"

SRHEN This is a better and simpler implementation for "SRHEN: Stepwise-Refining Homography Estimation Network via Parsing Geometric Correspondences in

Jan 18, 2022
Geometric Algebra package for JAX

JAXGA - JAX Geometric Algebra GitHub | Docs JAXGA is a Geometric Algebra package on top of JAX. It can handle high dimensional algebras by storing onl

Jun 5, 2022
Geometric Vector Perceptrons --- a rotation-equivariant GNN for learning from biomolecular structure
Geometric Vector Perceptrons --- a rotation-equivariant GNN for learning from biomolecular structure

Geometric Vector Perceptron Implementation of equivariant GVP-GNNs as described in Learning from Protein Structure with Geometric Vector Perceptrons b

Jul 5, 2022
Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer)
Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer)

Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer) Introduction By applying the

Dec 17, 2021
GeoTransformer - Geometric Transformer for Fast and Robust Point Cloud Registration
GeoTransformer - Geometric Transformer for Fast and Robust Point Cloud Registration

Geometric Transformer for Fast and Robust Point Cloud Registration PyTorch imple

Jul 5, 2022
Buffon’s needle: one of the oldest problems in geometric probability

Buffon-s-Needle Buffon’s needle is one of the oldest problems in geometric proba

Feb 18, 2022
Contains code for Deep Kernelized Dense Geometric Matching

DKM - Deep Kernelized Dense Geometric Matching Contains code for Deep Kernelized Dense Geometric Matching We provide pretrained models and code for ev

Jun 27, 2022