Ludwig is a toolbox that allows to train and evaluate deep learning models without the need to write code.

Ludwig logo

PyPI version Downloads Build Status License FOSSA Status CII Best Practices

Translated in πŸ‡°πŸ‡· Korean/

Ludwig is a toolbox that allows users to train and test deep learning models without the need to write code. It is built on top of TensorFlow.

To train a model you need to provide is a file containing your data, a list of columns to use as inputs, and a list of columns to use as outputs, Ludwig will do the rest. Simple commands can be used to train models both locally and in a distributed way, and to use them to predict new data.

A programmatic API is also available to use Ludwig from Python. A suite of visualization tools allows you to analyze models' training and test performance and to compare them.

Ludwig is built with extensibility principles in mind and is based on datatype abstractions, making it easy to add support for new datatypes as well as new model architectures.

It can be used by practitioners to quickly train and test deep learning models as well as by researchers to obtain strong baselines to compare against and have an experimentation setting that ensures comparability by performing the same data processing and evaluation.

Ludwig provides a set of model architectures that can be combined together to create an end-to-end model for a given use case. As an analogy, if deep learning libraries provide the building blocks to make your building, Ludwig provides the buildings to make your city, and you can choose among the available buildings or add your own building to the set of available ones.

The core design principles baked into the toolbox are:

  • No coding required: no coding skills are required to train a model and use it for obtaining predictions.
  • Generality: a new datatype-based approach to deep learning model design makes the tool usable across many different use cases.
  • Flexibility: experienced users have extensive control over model building and training, while newcomers will find it easy to use.
  • Extensibility: easy to add new model architecture and new feature datatypes.
  • Understandability: deep learning model internals are often considered black boxes, but Ludwig provides standard visualizations to understand their performance and compare their predictions.
  • Open Source: Apache License 2.0

LF AI & Data logo

Ludwig is hosted by the Linux Foundation as part of the LF AI & Data Foundation. For details about who's involved and how Ludwig fits into the larger open source AI landscape, read the Linux Foundation announcement.

Installation

Ludwig requires you to use Python 3.6+. If you don’t have Python 3 installed, install it by running:

sudo apt install python3  # on ubuntu
brew install python3      # on mac

You may want to use a virtual environment to maintain an isolated Python environment.

virtualenv -p python3 venv

In order to install Ludwig just run:

pip install ludwig

This will install only Ludwig's basic requirements, different feature types require different dependencies. We divided them as different extras so that users could install only the ones they actually need:

  • ludwig[text] for text dependencies.
  • ludwig[audio] for audio and speech dependencies.
  • ludwig[image] for image dependencies.
  • ludwig[hyperopt] for hyperparameter optimization dependencies.
  • ludwig[horovod] for distributed training dependencies.
  • ludwig[serve] for serving dependencies.
  • ludwig[viz] for visualization dependencies.
  • ludwig[test] for dependencies needed for testing.

Distributed training is supported with Horovod, which can be installed with pip install ludwig[horovod] or HOROVOD_GPU_OPERATIONS=NCCL pip install ludwig[horovod] for GPU support. See Horovod's installation guide for full details on available installation options.

Any combination of extra packages can be installed at the same time with pip install ludwig[extra1,extra2,...] like for instance pip install ludwig[text,viz]. The full set of dependencies can be installed with pip install ludwig[full].

For developers who wish to build the source code from the repository:

git clone [email protected]:ludwig-ai/ludwig.git
cd ludwig
virtualenv -p python3 venv
source venv/bin/activate
pip install -e '.[test]'

Note: that if you are running without GPUs, you may wish to use the CPU-only version of TensorFlow, which takes up much less space on disk. To use a CPU-only TensorFlow version, uninstall tensorflow and replace it with tensorflow-cpu after having installed ludwig. Be sure to install a version within the compatible range as shown in requirements.txt.

Basic Principles

Ludwig provides three main functionalities: training models and using them to predict and evaluate them. It is based on datatype abstraction, so that the same data preprocessing and postprocessing will be performed on different datasets that share datatypes and the same encoding and decoding models developed can be re-used across several tasks.

Training a model in Ludwig is pretty straightforward: you provide a dataset file and a config YAML file.

The config contains a list of input features and output features, all you have to do is specify names of the columns in the dataset that are inputs to your model alongside with their datatypes, and names of columns in the dataset that will be outputs, the target variables which the model will learn to predict. Ludwig will compose a deep learning model accordingly and train it for you.

Currently, the available datatypes in Ludwig are:

  • binary
  • numerical
  • category
  • set
  • bag
  • sequence
  • text
  • timeseries
  • image
  • audio
  • date
  • h3
  • vector

By choosing different datatype for inputs and outputs, users can solve many different tasks, for instance:

  • text input + category output = text classifier
  • image input + category output = image classifier
  • image input + text output = image captioning
  • audio input + binary output = speaker verification
  • text input + sequence output = named entity recognition / summarization
  • category, numerical and binary inputs + numerical output = regression
  • timeseries input + numerical output = forecasting model
  • category, numerical and binary inputs + binary output = fraud detection

take a look at the Examples to see how you can use Ludwig for several more tasks.

The config can contain additional information, in particular how to preprocess each column in the data, which encoder and decoder to use for each one, architectural and training parameters, hyperparameters to optimize. This allows ease of use for novices and flexibility for experts.

Training

For example, given a text classification dataset like the following:

doc_text class
Former president Barack Obama ... politics
Juventus hired Cristiano Ronaldo ... sport
LeBron James joins the Lakers ... sport
... ...

you want to learn a model that uses the content of the doc_text column as input to predict the values in the class column. You can use the following config:

{input_features: [{name: doc_text, type: text}], output_features: [{name: class, type: category}]}

and start the training typing the following command in your console:

ludwig train --dataset path/to/file.csv --config "{input_features: [{name: doc_text, type: text}], output_features: [{name: class, type: category}]}"

where path/to/file.csv is the path to a UTF-8 encoded CSV file containing the dataset in the previous table (many other data formats are supported). Ludwig will:

  1. Perform a random split of the data.
  2. Preprocess the dataset.
  3. Build a ParallelCNN model (the default for text features) that decodes output classes through a softmax classifier.
  4. Train the model on the training set until the performance on the validation set stops improving.

Training progress will be displayed in the console, but the TensorBoard can also be used.

If you prefer to use an RNN encoder and increase the number of epochs to train for, all you have to do is to change the config to:

{input_features: [{name: doc_text, type: text, encoder: rnn}], output_features: [{name: class, type: category}], training: {epochs: 50}}

Refer to the User Guide to find out all the options available to you in the config and take a look at the Examples to see how you can use Ludwig for several different tasks.

After training, Ludwig will create a results directory containing the trained model with its hyperparameters and summary statistics of the training process. You can visualize them using one of the several visualization options available in the visualize tool, for instance:

ludwig visualize --visualization learning_curves --training_statistics path/to/training_statistics.json

This command will display a graph like the following, where you can see loss and accuracy during the training process:

Learning Curves

Several more visualizations are available, please refer to Visualizations for more details.

Distributed Training

You can distribute the training of your models using Horovod, which allows training on a single machine with multiple GPUs as well as on multiple machines with multiple GPUs. Refer to the User Guide for full details.

Prediction and Evaluation

If you want your previously trained model to predict target output values on new data, you can type the following command in your console:

ludwig predict --dataset path/to/data.csv --model_path /path/to/model

Running this command will return model predictions.

If your dataset also contains ground truth values of the target outputs, you can compare them to the predictions obtained from the model to evaluate the model performance.

ludwig evaluate --dataset path/to/data.csv --model_path /path/to/model

This will produce evaluation performance statistics that can be visualized by the visualize tool, which can also be used to compare performances and predictions of different models, for instance:

ludwig visualize --visualization compare_performance --test_statistics path/to/test_statistics_model_1.json path/to/test_statistics_model_2.json

will return a bar plot comparing the models on different metrics:

Performance Comparison

A handy ludwig experiment command that performs training and prediction one after the other is also available.

Programmatic API

Ludwig also provides a simple programmatic API that allows you to train or load a model and use it to obtain predictions on new data:

from ludwig.api import LudwigModel

# train a model
config = {...}
model = LudwigModel(config)
train_stats = model.train(training_data)

# or load a model
model = LudwigModel.load(model_path)

# obtain predictions
predictions = model.predict(test_data)

config containing the same information of the YAML file provided to the command line interface. More details are provided in the User Guide and in the API documentation.

Extensibility

Ludwig is built from the ground up with extensibility in mind. It is easy to add an additional datatype that is not currently supported by adding a datatype-specific implementation of abstract classes that contain functions to preprocess the data, encode it, and decode it.

Furthermore, new models, with their own specific hyperparameters, can be easily added by implementing a class that accepts tensors (of a specific rank, depending on the datatype) as inputs and provides tensors as output. This encourages reuse and sharing new models with the community. Refer to the Developer Guide for further details.

Full documentation

You can find the full documentation here.

License

FOSSA Status

Owner
Ludwig
Ludwig is a toolbox that allows to train and test deep learning models without the need to write code. It is an incubation level project in LF AI Foundation.
Ludwig
Comments
  • TF2 porting: Sequence generator decoder

    TF2 porting: Sequence generator decoder

    Code Pull Requests

    Start of the sequence feature's generator decoder. This is not a complete implementation. I wanted to give you an early look at the approach I'm taking.

    Instead of retrofitting the TF1 code, I'm using the approach described in TFA and demonstrated in these examples: example one and example two

    I'm just restating the obvious to establish my starting point. Ludwig supports one of two tensor shapes for Decoder input

    • shape [b, h] w/o attention
    • shape [b, s, h] with attention

    The expected decoder output: shape [b, s, c]

    One question, it is not clear to me how to determine sequence length for the decoder's output tensor. Right now I'm just using num_classes as a place holder so that the decoder generates a rank 3 tensor.

    State of the implementation

    • Initial commit supports decoder input shape [b, h] and generates a tensor with shape [b, s, c], implemented the basic decoder.
    • No exceptions are generated infull_experiment(). The loss value decreases each epoch.
    • These warning messages appear during training. Same as with category feature.
    WARNING:tensorflow:Gradients do not exist for variables ['embeddings:0', 'ecd/sequence_output_feature/fc_stack/fc_layer/dense/kernel:0', 'ecd/sequence_output_feature/fc_stack/fc_layer/dense/bias:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_1/dense_1/kernel:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_1/dense_1/bias:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_2/dense_2/kernel:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_2/dense_2/bias:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_3/dense_3/kernel:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_3/dense_3/bias:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_4/dense_4/kernel:0', 'ecd/sequence_output_feature/fc_stack/fc_layer_4/dense_4/bias:0'] when minimizing the loss.
    

    Next Steps

    • Add support for Attention
  • TF2 porting: eager mode training and evaluation, numerical and binary features

    TF2 porting: eager mode training and evaluation, numerical and binary features

    Reopening PR for TF2 porting...

    I'm hoping this posting provides some evidence of progress. With your last guidance, I was able to get a "minimal training loop" working with model subclassing and eager execution.

    I wanted to offer you an early look at how I'm adapting Ludwig training to TF2 eager execution.

    This commit demonstrates the minimal training loop for this model_definition:

    input_features = [
        {'name': 'x1', 'type': 'numerical', 'preprocessing': {'normalization': 'zscore'}},
        {'name': 'x2', 'type': 'numerical', 'preprocessing': {'normalization': 'zscore'}},
        {'name': 'x3', 'type': 'numerical', 'preprocessing': {'normalization': 'zscore'}}
    ]
    output_features = [
        {'name': 'y', 'type': 'numerical'}
    ]
    
    model_definition = {
        'input_features': input_features,
        'output_features': output_features,
        'combiner': {
            'type': 'concat',  # 'concat',
            'num_fc_layers': 5,
            'fc_size': 64
        },
        'training': {'epochs': 100}
    }
    

    The main result of this "minimal training loop" is demonstrating:

    • not blowing up while running the specified number epochs
    • Create numpy arrays used for training from input features and output features specified in model_definition
    • reduction in the loss function during training

    Here is an excerpt of the log file for the minimal training loop:

    
    Epoch   1
    
    Epoch   1
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 50.40it/s]
    Epoch 1, train Loss: 9732.708984375, : train metric 9743.146484375
    
    Epoch   2
    
    Epoch   2
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 77.71it/s]
    Epoch 2, train Loss: 9711.203125, : train metric 9721.9404296875
    
    Epoch   3
    
    Epoch   3
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 80.07it/s]
    Epoch 3, train Loss: 9678.2998046875, : train metric 9689.5517578125
    
    <<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>
    
    Epoch  50
    
    Epoch  50
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 80.76it/s]
    Epoch 50, train Loss: 1540.60400390625, : train metric 1547.77294921875
    
    Epoch  51
    
    Epoch  51
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 75.30it/s]
    Epoch 51, train Loss: 1510.5498046875, : train metric 1517.5792236328125
    
    Epoch  52
    
    Epoch  52
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 75.37it/s]
    Epoch 52, train Loss: 1481.64404296875, : train metric 1488.5391845703125
    
    <<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    
    Epoch  98
    
    Epoch  98
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 78.33it/s]
    Epoch 98, train Loss: 787.4824829101562, : train metric 791.1544189453125
    
    Epoch  99
    
    Epoch  99
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 75.26it/s]
    Epoch 99, train Loss: 779.5379028320312, : train metric 783.1728515625
    
    Epoch 100
    
    Epoch 100
    Training: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:00<00:00, 83.06it/s]
    Epoch 100, train Loss: 771.7520141601562, : train metric 775.3506469726562
    
    

    Here is the entire eager execution log file: proof-of-concept_log_tf2_eager_exec.txt.

    Now comes all the limitations and caveats for the current state of the code:

    • training loss reduction is not has fast as the TF1 implementation. Here is the TF1 log file for the same model proof-of-concept_log_tf1.txt.
    • Commented out large sections of the code in the model.train() method. I envision re-enabling and modifying the commented code as work progresses
    • Several items were hard-coded in this implemantion to miminize the amount of change required just to demonstrate the "training loop". The hard-code functions are
      • The encoder structure is hard-coded in the model.call() method. This will change to reflect the encorder/decoders specified in the model_definition
      • the objective, loss and metric functions are hard-coded. Future work will be build these from the model_defintion
    • One of the sections of code commented out creates these data structures. Without theses structures Ludwig processing after the training loop abnormally terminates. This will be fixed as the work progresses. Actually, I'm thinking that this is the next thing to fix with the simple model I'm using for testing. If I can get these enabled, then the rest of Ludwig processing "should work".
      • progress_tracker.train_stats
      • progress_tracker.vali_stats
      • progress_tracker.test_stats
  • TF2 porting: Enable early stopping + model save and load

    TF2 porting: Enable early stopping + model save and load

    Code Pull Requests

    Re-introduced early stopping option. From what I can tell there is no existing unit test for early stopping; I added early such a test. This test used two values (3 and 5) for early_stop option and confirmed training stopped per the early stop specification.

    pytest -v test_model_training_options.py
    ======================================================= test session starts ========================================================
    platform linux -- Python 3.6.9, pytest-5.4.3, py-1.8.1, pluggy-0.13.1 -- /usr/bin/python3
    cachedir: .pytest_cache
    rootdir: /opt/project
    plugins: pycharm-0.6.0, typeguard-2.8.0
    collected 2 items
    
    test_model_training_options.py::test_early_stopping[3] PASSED                                                                [ 50%]
    test_model_training_options.py::test_early_stopping[5] PASSED                                                                [100%]
    
    ========================================================= warnings summary =========================================================
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py:15
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
        import imp
    
    -- Docs: https://docs.pytest.org/en/latest/warnings.html
    ================================================== 2 passed, 1 warning in 12.85s ===================================================
    

    The test_model_training_options.py can be used as a foundation to other unit tests around model training options, e.g. save or not save various logs, etc.

    This PR enabled only the early stopping test. Nothing else was reenabled.

    If this looks OK, I can start re-enabling other model training options.

  • TF2 porting: initial work

    TF2 porting: initial work

    @w4nderlust This is a test of sending results of my work. Initially I tried pushing my changes directly to uber/ludwig but it failed because I did not have write permission to uber/ludwig.

    I'm now trying to submit my changes through a PR for branch tf2_porting branch in my forkjimthompson5802/ludwig. The target for this PR is uber/ludwig' branch 'tf2_porting. If this works, then I can just add commits to this PR.

    The Docker image I use for my development environment is built with the updated requirements.txt on the tf2_porting branch, which contains

    tensorflow==2.1
    tensorflow-addons
    

    With this initial set of commits, train completes and some, not all, of the data are saved for TensorBoard. For my test, I'm using the Titanic Survivor example. Here is the log from training. tf2_sample_train_log.txt

    Here is screenshot of TensorBoard for the data that was collected. Screen Shot 2020-02-15 at 01 12 30

    Let me know what you think.

  • TF2 porting: sequence feature

    TF2 porting: sequence feature

    Code Pull Requests

    This is the start of porting sequence feature to TF2. There is still more to be done. Current state of code:

    • placed TF2 related stub methods in sequence_feature.py. These will be filled in as work progresses.
    • Created classes SequencePassthroughEncoder and SequenceEmbedEncoder as subclass of tf.keras.layers.Layer. Integrated existing functions to work with the Layer subclass.

    From a small test both encoders work. The test passed an input array of shape [2, 5] through the SequencePassthroughEncoder and SequenceEmbedEncoder. Results of the test:

    • Passthrough encoder with reduce_output=None resulted in an output tensor with shape [2, 5, 1]
    • Embed encoder with embedding_size=3, reduce_output='sum' resulted in an output tensor with shape [2,3]
    • Embed encoder with embedding_size=3, reduce_output='None' resulted in an output tensor with shape [2, 5, 3]

    Does this look correct?

    Next steps for me is to get a working decoder and full_experiment() to complete successfully for a simple model.

    This is the simple test case and its output.

    import numpy as np
    import tensorflow as tf
    
    from ludwig.models.modules.sequence_encoders import SequencePassthroughEncoder, \
        SequenceEmbedEncoder
    
    # setup input
    input = np.array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]])
    
    # test Passthrough encoder
    print("Sample SequencePassthroughEncoder")
    spt = SequencePassthroughEncoder()
    output = spt(input)
    print('input shape:', input.shape,'\nvalues:\n', input)
    print('\noutput shape:\n', output.shape,'\nvalues\n', output.numpy())
    
    
    # test embed encoder
    print("\n\nSample SequenceEmbedEncoder")
    vocab = list('abcdefghij')
    print('vocab:', len(vocab), vocab)
    emb = SequenceEmbedEncoder(vocab, embedding_size=3)
    output2 = emb(input)
    print('input shape:', input.shape,'\nvalues:\n', input)
    print('\noutput shape:\n', output2.shape,'\nvalues\n', output2.numpy())
    
    
    # test embed encoder with reduce_output=None
    print("\n\nSample SequenceEmbedEncoder, reduce_output=None")
    vocab = list('abcdefghij')
    print('vocab:', len(vocab), vocab)
    emb2 = SequenceEmbedEncoder(vocab, embedding_size=3, reduce_output='None')
    output3 = emb2(input)
    print('input shape:', input.shape,'\nvalues:\n', input)
    print('\noutput shape:\n', output3.shape,'\nvalues\n', output3.numpy())
    

    Test output

    2516c6abd7cc:python -u /opt/project/sandbox/tf2_port/sequence_encoder_tester.py
    2020-04-10 03:41:27.645738: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
    2020-04-10 03:41:27.645874: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
    2020-04-10 03:41:27.645895: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
    Sample SequencePassthroughEncoder
    2020-04-10 03:41:28.838864: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
    2020-04-10 03:41:28.838974: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
    2020-04-10 03:41:28.839016: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (2516c6abd7cc): /proc/driver/nvidia/version does not exist
    2020-04-10 03:41:28.839430: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2020-04-10 03:41:28.845809: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2791435000 Hz
    2020-04-10 03:41:28.846760: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x439f600 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
    2020-04-10 03:41:28.846828: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    input shape: (2, 5) 
    values:
     [[0 1 2 3 4]
     [5 6 7 8 9]]
    
    output shape:
     (2, 5, 1) 
    values
     [[[0.]
      [1.]
      [2.]
      [3.]
      [4.]]
    
     [[5.]
      [6.]
      [7.]
      [8.]
      [9.]]]
    
    
    Sample SequenceEmbedEncoder
    vocab: 10 ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
    input shape: (2, 5) 
    values:
     [[0 1 2 3 4]
     [5 6 7 8 9]]
    
    output shape:
     (2, 3) 
    values
     [[ 0.43313265  2.5553155   0.4227686 ]
     [-2.2676423   0.80896974 -0.28734756]]
    
    
    Sample SequenceEmbedEncoder, reduce_output=None
    vocab: 10 ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
    input shape: (2, 5) 
    values:
     [[0 1 2 3 4]
     [5 6 7 8 9]]
    
    output shape:
     (2, 5, 3) 
    values
     [[[ 0.          0.         -0.        ]
      [-0.3263557  -0.9351046   0.81713986]
      [ 0.10236788 -0.8854079   0.736264  ]
      [ 0.6721873  -0.6389749  -0.22343826]
      [ 0.9717505   0.37574863 -0.77134395]]
    
     [[-0.29198527  0.8138087  -0.12796235]
      [-0.7267971   0.70457935  0.5321882 ]
      [ 0.8782997   0.01055479  0.969434  ]
      [ 0.89878535  0.36403346  0.67668414]
      [ 0.55935645 -0.59581757  0.71213484]]]
    
    Process finished with exit code 0
    
  • TF2 porting: category feature

    TF2 porting: category feature

    Code Pull Requests

    Here is the start of converting the category feature to TF2 eager execution. More work is needed to complete.

    The following have been implemented:

    • Setup category encoder and decoder
    • Adapt the category encoder to use current Ludwig Embed class
    • Custom softmax cross entropy loss function for training and evaluation
    • Custom softmax cross entropy metric function

    At this point training phase completes w/o error. As noted above only the Loss function has been implemented.

    The predict phase fails because of an incomplete implementation of the predictions() method and missing metric functions. The work to be done is similar to what I did with the binary feature.

    Since this is the first time I've implemented encoder and decoder, I'd appreciate if you would take a look at how I implemented them.

    I'm attaching the training data I'm using for testing. Included in the zip file is a log file from a test run. ludwig_category_feature.zip

    Here is the model definition I'm using

    python -m ludwig.experiment --data_csv data4/train.csv \
      --skip_save_processed_input \
      --model_definition "{input_features: [{name: x1, type: numerical, preprocessing: {normalization: minmax}},
        {name: x2, type: numerical, preprocessing: {normalization: zscore}},
        {name: x3, type: numerical, preprocessing: {normalization: zscore}},
        {name: c1, type: category, embedding_size: 6}],
        combiner: {type: concat, num_fc_layers: 5, fc_size: 64},
        output_features: [{name: y, type: category}], training: {epochs: 10}}"
    
  • Initial pass at wandb Ludwig integration

    Initial pass at wandb Ludwig integration

    This is initial pass at a basic integration of wandb with Ludwig. It currently mirrors tensorboard events, stores hyperparameters, syncs artifacts, and stores evaluation results.

    I'm using the python "black" code formatter which seems to be clashing with some of the existing code. Is there a preferred formatter to use?

  • Mac M1 Support

    Mac M1 Support

    Hi, I'm wondering if Ludwig currently has support for the Mac M1? I have tried to install it several times through the steps on the website and no luck.

    I've downloaded Tensorflow 2.4.0-rc0 (the only one available for M1) separately as it wasn't getting anywhere through the "pip install ludwig" command. I kept getting dependency conflict errors and I corrected them for the most part but there seems to be no workaround for TF- see conflicts below.

    The conflict is caused by: ludwig 0.3.3 depends on tensorflow>=2.3.1 ludwig 0.3.2 depends on tensorflow>=2.3.1 ludwig 0.3.1 depends on tensorflow>=2.2 ludwig 0.3 depends on tensorflow>=2.2 ludwig 0.2.2.8 depends on tensorflow==1.15.3 ludwig 0.2.2.7 depends on tensorflow==1.15.3 ludwig 0.2.2.6 depends on tensorflow==1.15.2 ludwig 0.2.2.5 depends on tensorflow==1.15.2 ludwig 0.2.2.4 depends on tensorflow==1.15.2 ludwig 0.2.2.3 depends on tensorflow==1.15.2 ludwig 0.2.2.2 depends on tensorflow-gpu==1.15.2 ludwig 0.2.2 depends on tensorflow-gpu==1.15.2 ludwig 0.2.1 depends on tensorflow==1.14.0 ludwig 0.2 depends on tensorflow==1.14.0 ludwig 0.1.2 depends on tensorflow==1.13.1 ludwig 0.1.1 depends on tensorflow==1.13.1 ludwig 0.1.0 depends on tensorflow>=1.12

    To fix this you could try to:

    1. loosen the range of package versions you've specified
    2. remove package versions to allow pip attempt to solve the dependency conflict

    It seems M1's limited TF availability is not letting the Ludwig install get through the TF dependencies.

    Expected behavior Successfully install Ludwig.

    Environment (please complete the following information):

    • OS: Big Sur 11.2.1
    • Python 3.8
    • Ludwig 0.3.3

    Thanks

  • pandas.errors.ParserError: Error tokenizing data. C error: Expected 83 fields in line 40, saw 92

    pandas.errors.ParserError: Error tokenizing data. C error: Expected 83 fields in line 40, saw 92

    Describe the bug Hi, I am getting this error message: pandas.errors.ParserError: Error tokenizing data. C error: Expected 83 fields in line 40, saw 92

    I made all the possible in order to use a good CSV, but I guess something is not fine. As a suggestion I believe it could be good to find a way to handle bad lines (skip) and present a value (total) of ignored lines.

    Back to my settings. I have UTF-8 for the saved csv file. My columns contain mostly text and some got ',' which I turned into '\t' as founded in the documentation. Also, I made the following basic test: I have re-imported the csv in Excel and made sure all the lines and columns (24 in total in my case) are in the right place. That was the case: all fine inside of MS Excel view.

    The YAML has been validated correctly. I exclude it is that one my case.

    Here below the lines from my terminal.

    ludwig_version: '0.1.0' command: ('/Library/Frameworks/Python.framework/Versions/3.6/bin/ludwig train ' '--data_csv /Users/my_mac/Projects/ML/LUDWIG/Wine/red_dataset.csv ' '--model_definition_file ' '/Users/my_mac/Projects/ML/LUDWIG/Wine/dataset.yaml') dataset_type: '/Users/my_mac/Projects/ML/LUDWIG/Wine/red_dataset.csv' model_definition: { 'combiner': {'type': 'concat'}, 'input_features': [ { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Order', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Produttore', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Tipo', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Descrizione', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Vitigni', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Vigneti', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Vinificazione', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Affinamento', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Filosofia', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Temperatura', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Quando_aprire', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Ideale', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Quando_bere', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Descrizione_long', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Colore', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Profumo', 'tied_weights': None, 'type': 'text'}, { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'Gusto', 'tied_weights': None, 'type': 'text'}, { 'in_memory': True, 'name': 'Immagine-src', 'should_resize': False, 'tied_weights': None, 'type': 'image'}], 'output_features': [ { 'dependencies': [], 'loss': { 'type': 'mean_squared_error', 'weight': 1}, 'name': 'Prezzo', 'reduce_dependencies': 'sum', 'reduce_input': 'sum', 'type': 'numerical', 'weight': 1}, { 'decoder': 'generator', 'dependencies': [], 'level': 'char', 'loss': { 'class_distance_temperature': 0, 'class_weights': 1, 'type': 'softmax_cross_entropy', 'weight': 1}, 'name': 'Denominazione', 'reduce_dependencies': 'sum', 'reduce_input': 'sum', 'type': 'text', 'weight': 1}, { 'decoder': 'generator', 'dependencies': [], 'level': 'char', 'loss': { 'class_distance_temperature': 0, 'class_weights': 1, 'type': 'softmax_cross_entropy', 'weight': 1}, 'name': 'Regione', 'reduce_dependencies': 'sum', 'reduce_input': 'sum', 'type': 'text', 'weight': 1}, { 'decoder': 'generator', 'dependencies': [], 'level': 'char', 'loss': { 'class_distance_temperature': 0, 'class_weights': 1, 'type': 'softmax_cross_entropy', 'weight': 1}, 'name': 'Gradazione', 'reduce_dependencies': 'sum', 'reduce_input': 'sum', 'type': 'text', 'weight': 1}], 'preprocessing': { 'bag': { 'fill_value': '', 'format': 'space', 'lowercase': 10000, 'missing_value_strategy': 'fill_with_const', 'most_common': False}, 'binary': { 'fill_value': 0, 'missing_value_strategy': 'fill_with_const'}, 'category': { 'fill_value': '', 'lowercase': False, 'missing_value_strategy': 'fill_with_const', 'most_common': 10000}, 'force_split': False, 'image': {'missing_value_strategy': 'backfill'}, 'numerical': { 'fill_value': 0, 'missing_value_strategy': 'fill_with_const'}, 'sequence': { 'fill_value': '', 'format': 'space', 'lowercase': False, 'missing_value_strategy': 'fill_with_const', 'most_common': 20000, 'padding': 'right', 'padding_symbol': '', 'sequence_length_limit': 256, 'unknown_symbol': ''}, 'set': { 'fill_value': '', 'format': 'space', 'lowercase': False, 'missing_value_strategy': 'fill_with_const', 'most_common': 10000}, 'split_probabilities': (0.7, 0.1, 0.2), 'stratify': None, 'text': { 'char_format': 'characters', 'char_most_common': 70, 'char_sequence_length_limit': 1024, 'fill_value': '', 'lowercase': True, 'missing_value_strategy': 'fill_with_const', 'padding': 'right', 'padding_symbol': '', 'unknown_symbol': '', 'word_format': 'space_punct', 'word_most_common': 20000, 'word_sequence_length_limit': 256}, 'timeseries': { 'fill_value': '', 'format': 'space', 'missing_value_strategy': 'fill_with_const', 'padding': 'right', 'padding_value': 0, 'timeseries_length_limit': 256}}, 'training': { 'batch_size': 128, 'bucketing_field': None, 'decay': False, 'decay_rate': 0.96, 'decay_steps': 10000, 'dropout_rate': 0.0, 'early_stop': 3, 'epochs': 10, 'gradient_clipping': None, 'increase_batch_size_on_plateau': 0, 'increase_batch_size_on_plateau_max': 512, 'increase_batch_size_on_plateau_patience': 5, 'increase_batch_size_on_plateau_rate': 2, 'learning_rate': 0.001, 'learning_rate_warmup_epochs': 5, 'optimizer': { 'beta1': 0.9, 'beta2': 0.999, 'epsilon': 1e-08, 'type': 'adam'}, 'reduce_learning_rate_on_plateau': 0, 'reduce_learning_rate_on_plateau_patience': 5, 'reduce_learning_rate_on_plateau_rate': 0.5, 'regularization_lambda': 0, 'regularizer': 'l2', 'staircase': False, 'validation_field': 'combined', 'validation_measure': 'loss'}}

    Using full raw csv, no hdf5 and json file with the same name have been found Building dataset (it may take a while) Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/utils/data_utils.py", line 46, in read_csv df = pd.read_csv(data_fp, header=header) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/io/parsers.py", line 678, in parser_f return _read(filepath_or_buffer, kwds) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/io/parsers.py", line 446, in _read data = parser.read(nrows) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/io/parsers.py", line 1036, in read ret = self._engine.read(nrows) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/io/parsers.py", line 1848, in read data = self._reader.read(nrows) File "pandas/_libs/parsers.pyx", line 876, in pandas._libs.parsers.TextReader.read File "pandas/_libs/parsers.pyx", line 891, in pandas._libs.parsers.TextReader._read_low_memory File "pandas/_libs/parsers.pyx", line 945, in pandas._libs.parsers.TextReader._read_rows File "pandas/_libs/parsers.pyx", line 932, in pandas._libs.parsers.TextReader._tokenize_rows File "pandas/_libs/parsers.pyx", line 2112, in pandas._libs.parsers.raise_parser_error pandas.errors.ParserError: Error tokenizing data. C error: Expected 83 fields in line 40, saw 92

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/bin/ludwig", line 11, in sys.exit(main()) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/cli.py", line 86, in main CLI() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/cli.py", line 64, in init getattr(self, args.command)() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/cli.py", line 70, in train train.cli(sys.argv[2:]) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/train.py", line 663, in cli full_train(**vars(args)) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/train.py", line 224, in full_train random_seed=random_seed File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/data/preprocessing.py", line 457, in preprocess_for_training random_seed=random_seed File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/data/preprocessing.py", line 54, in build_dataset dataset_df = read_csv(dataset_csv) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ludwig/utils/data_utils.py", line 48, in read_csv logging.WARNING('Failed to parse the CSV with pandas default way,' TypeError: 'int' object is not callable

  • How to get the visualization  result?

    How to get the visualization result?

    After running ludwig visualize --visualization learning_curves -ts results/experiment_run_4/training_statistics.json, it returns empty. Where to get the visualization results?Is it saved as a image in somewhere?

  • Save models with SavedModel

    Save models with SavedModel

    Hello , I know there is #55 issue for exposing the model . i have read the issue . it was suggested that you can use a ludwig developed model in tensorflow serving . I have been trying to convert it to in a format so i can use it in tensorflow serving.

    For example , i have followed , readallcat.csv example and develped model in ludwig which is in the format ,

    checkpoint model_weights.data-00000-of-00001 model_weights.meta model_hyperparameters.json model_weights.index train_set_metadata.json

    Now for tensorflow serving the model needs to be in the below format as mentioned https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md ,

    assets/ variables/ variables.data-***-of-*** variables.index saved_model.pb

    I have been trying to use tensorflow "SavedModel api" , but no luck for the conversion . if you please provide some guidance will be a big help . Thank you

  • Set default validation metrics based on the output feature type.

    Set default validation metrics based on the output feature type.

    Reduced version of #2811

    By default, set validation_metric to the output feature type's default_validation_metric. If there are multiple output features, use validation_field=combined,validation_metric=loss`.

    Update schema docstrings for validation_field and validation metric.

  • [Annotations] Encoder and Decoder Registries

    [Annotations] Encoder and Decoder Registries

    This PR creates accessor methods for encoder and decoder registries and uses the DeveloperAPI annotation. This will be the best way to retrieve these registries in the future.

    I've made the original variable paths private because there's no good way to remove them without breaking the logic where we instantiate a registry and incrementally register things.

  • [Annotations] Move feature registries into accessor functions

    [Annotations] Move feature registries into accessor functions

    This PR moves base_type_registry, input_type_registry and output_type_registry into accessor methods so that they can't be accessed directly. It also annotates these accessor methods with the DeveloperAPI annotation.

  • [IN PROGRESS] By default, set validation_metric to the output feature type's default validation metric.

    [IN PROGRESS] By default, set validation_metric to the output feature type's default validation metric.

    Changes:

    • Deprecate validate_config in schema/__init__.py, and move config validation to ludwig/schema/model_config.ModelConfig, and perform validation on top of a comprehensive config.
    • Introduce auxiliary validations that checks config errors that are not easily covered by marshmallow schemas like parameter interdependencies. As these are built into schemas, auxiliary validations can be gradually removed.
    • By default, set validation_metric to the output feature type's default_validation_metric. If there are multiple output features, use validation_field=combined, validation_metric=loss.
    • Update docstrings for validation_field and validation metric in the schema
  • WIP AVMnist dataset

    WIP AVMnist dataset

    Code Pull Requests

    Please provide the following:

    • The initial iteration of the av_mnist dataset using the torch datasetLoader like structures
    • https://github.com/ludwig-ai/ludwig/issues/2638
    • a test is included but very much a WIP

    Documentation Pull Requests

    Will be added once this PR is complete

a delightful machine learning tool that allows you to train, test and use models without writing code
a delightful machine learning tool that allows you to train, test and use models without writing code

igel A delightful machine learning tool that allows you to train/fit, test and use models without writing code Note I'm also working on a GUI desktop

Dec 2, 2022
HeatNet is a python package that provides tools to build, train and evaluate neural networks designed to predict extreme heat wave events globally on daily to subseasonal timescales.

HeatNet HeatNet is a python package that provides tools to build, train and evaluate neural networks designed to predict extreme heat wave events glob

Jul 7, 2022
Train/evaluate a Keras model, get metrics streamed to a dashboard in your browser.
Train/evaluate a Keras model, get metrics streamed to a dashboard in your browser.

Hera Train/evaluate a Keras model, get metrics streamed to a dashboard in your browser. Setting up Step 1. Plant the spy Install the package pip

Nov 19, 2022
This repository contains a set of codes to run (i.e., train, perform inference with, evaluate) a diarization method called EEND-vector-clustering.

EEND-vector clustering The EEND-vector clustering (End-to-End-Neural-Diarization-vector clustering) is a speaker diarization framework that integrates

Nov 24, 2022
Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation.

============================================================================================================ `MILA will stop developing Theano <https:

Dec 3, 2022
Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation.

============================================================================================================ `MILA will stop developing Theano <https:

Dec 3, 2022
Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation.

============================================================================================================ `MILA will stop developing Theano <https:

Feb 12, 2021
OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

OCTIS : Optimizing and Comparing Topic Models is Simple! OCTIS (Optimizing and Comparing Topic models Is Simple) aims at training, analyzing and compa

Nov 30, 2022
Narya API allows you track soccer player from camera inputs, and evaluate them with an Expected Discounted Goal (EDG) Agent

Narya The Narya API allows you track soccer player from camera inputs, and evaluate them with an Expected Discounted Goal (EDG) Agent. This repository

Nov 26, 2022
A framework that allows people to write their own Rocket League bots.
A framework that allows people to write their own Rocket League bots.

YOU PROBABLY SHOULDN'T PULL THIS REPO Bot Makers Read This! If you just want to make a bot, you don't need to be here. Instead, start with one of thes

Nov 27, 2022
Code for paper " AdderNet: Do We Really Need Multiplications in Deep Learning?"
Code for paper

AdderNet: Do We Really Need Multiplications in Deep Learning? This code is a demo of CVPR 2020 paper AdderNet: Do We Really Need Multiplications in De

Dec 5, 2022
TLDR; Train custom adaptive filter optimizers without hand tuning or extra labels.
TLDR; Train custom adaptive filter optimizers without hand tuning or extra labels.

AutoDSP TLDR; Train custom adaptive filter optimizers without hand tuning or extra labels. About Adaptive filtering algorithms are commonplace in sign

Sep 19, 2022
This tool uses Deep Learning to help you draw and write with your hand and webcam.
This tool uses Deep Learning to help you draw and write with your hand and webcam.

This tool uses Deep Learning to help you draw and write with your hand and webcam. A Deep Learning model is used to try to predict whether you want to have 'pencil up' or 'pencil down'.

Nov 30, 2022
BasicRL: easy and fundamental codes for deep reinforcement learning。It is an improvement on rainbow-is-all-you-need and OpenAI Spinning Up.
BasicRL: easy and fundamental codes for deep reinforcement learning。It is an improvement on rainbow-is-all-you-need and OpenAI Spinning Up.

BasicRL: easy and fundamental codes for deep reinforcement learning BasicRL is an improvement on rainbow-is-all-you-need and OpenAI Spinning Up. It is

Apr 28, 2022
Metrics to evaluate quality and efficacy of synthetic datasets.
Metrics to evaluate quality and efficacy of synthetic datasets.

An Open Source Project from the Data to AI Lab, at MIT Metrics for Synthetic Data Generation Projects Website: https://sdv.dev Documentation: https://

Nov 19, 2022