提交测试
20
docs/Makefile
Normal file
@@ -0,0 +1,20 @@
|
||||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line, and also
|
||||
# from the environment for the first two.
|
||||
SPHINXOPTS ?=
|
||||
SPHINXBUILD ?= sphinx-build
|
||||
SOURCEDIR = .
|
||||
BUILDDIR = _build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
152
docs/README.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Documenting
|
||||
|
||||
This guide is for developers who write API documentation. To build the documentation, run
|
||||
|
||||
`pip install -r tools/doc_requirements.txt` to install the dependencies for documentation.
|
||||
|
||||
Then, run
|
||||
|
||||
```make html``` on Linux
|
||||
|
||||
```make.bat html``` on Windows
|
||||
|
||||
## Documenting Python API
|
||||
|
||||
The best way to document our Python API is to do so directly in the code. That way it's always extracted from a location
|
||||
where it's closest to the actual code and most likely to be correct.
|
||||
|
||||
Instead of using the older and more cumbersome restructredText Docstring specification we have adopted the more
|
||||
streamlined [Google Python Style Docstring][#5] format. This is how you would document an API function in Python:
|
||||
|
||||
```python
|
||||
|
||||
def answer_question(question):
|
||||
"""This function can answer some questions.
|
||||
|
||||
It currently only answers a limited set of questions so don't expect it to know everything.
|
||||
|
||||
Args:
|
||||
question (str): The question passed to the function, trailing question mark is not necessary and
|
||||
casing is not important.
|
||||
|
||||
Returns:
|
||||
answer (str): The answer to the question or ``None`` if it doesn't know the answer.
|
||||
|
||||
"""
|
||||
if question.lower().startswith("what is the answer to life, universe, and everything"):
|
||||
return str(42)
|
||||
else:
|
||||
return None
|
||||
```
|
||||
|
||||
After running the documentation generation system we will get this as the output:
|
||||
|
||||

|
||||
|
||||
One note:
|
||||
|
||||
The high-level structure is essentially in four parts:
|
||||
* A one-liner describing the function (without details or corner cases)
|
||||
* A paragraph that gives more detail on the function behavior (if necessary)
|
||||
* An `Args:` section (if the function takes arguments, note that `self` is not considered an argument)
|
||||
* A `Returns:` section (if the function can return somethings other than `None`)
|
||||
|
||||
We want to draw you attention to the following:
|
||||
|
||||
Indentation is key when writing docstrings. The documentation system is clever enough to remove uniform indentation.
|
||||
That is, as long as all the lines have the same amount of padding that padding will be ignored and not passed onto the restructured text processor. Fortunately clang-format leaves this funky formatting alone - respecting the raw string qualifier.
|
||||
|
||||
Let's now turn our attention to how we document modules and their attributes. We should of course only document
|
||||
modules that are part of our API (not internal helper modules) and only public attributes. Below is a detailed example:
|
||||
|
||||
```python
|
||||
"""Example of Google style docstrings for module.
|
||||
|
||||
This module demonstrates documentation as specified by the `Google Python
|
||||
Style Guide`_. Docstrings may extend over multiple lines. Sections are created
|
||||
with a section header and a colon followed by a block of indented text.
|
||||
|
||||
Example:
|
||||
Examples can be given using either the ``Example`` or ``Examples``
|
||||
sections. Sections support any reStructuredText formatting, including
|
||||
literal blocks::
|
||||
|
||||
$ python example.py
|
||||
|
||||
Section breaks are created by resuming unindented text. Section breaks
|
||||
are also implicitly created anytime a new section starts.
|
||||
|
||||
Attributes:
|
||||
module_level_variable1 (torch.Tensor): Module level variables may be documented in
|
||||
either the ``Attributes`` section of the module docstring, or in an
|
||||
inline docstring immediately following the variable.
|
||||
|
||||
Either form is acceptable, but the two should not be mixed. Choose
|
||||
one convention to document module level variables and be consistent
|
||||
with it.
|
||||
|
||||
module_level_variable3 (int, optional): An optional variable.
|
||||
|
||||
|
||||
Todo:
|
||||
* For module TODOs if you want them
|
||||
* These can be useful if you want to communicate any shortcomings in the module we plan to address
|
||||
|
||||
.. _Google Python Style Guide:
|
||||
http://google.github.io/styleguide/pyguide.html
|
||||
|
||||
"""
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
|
||||
module_level_variable1 = torch.tensor([12345])
|
||||
module_level_variable2 = np.array([12345])
|
||||
|
||||
"""np.ndarray: Module level variable documented inline. This approach may be preferable since it keeps the documentation closer to the code and the default
|
||||
assignment is shown. A downside is that the variable will get alphabetically sorted among functions in the module
|
||||
so won't have the same cohesion as the approach above."""
|
||||
|
||||
module_level_variable3 = None
|
||||
```
|
||||
|
||||
This is what the documentation would look like:
|
||||
|
||||

|
||||
|
||||
As we have mentioned we should not mix the `Attributes:` style of documentation with inline documentation of attributes.
|
||||
Notice how `module_level_variable3` appears in a separate block from all the other attributes that were documented. It
|
||||
is even after the TODO section. Choose one approach for your module and stick to it. There are valid reasons to pick
|
||||
one style above the other but don't cross the streams!
|
||||
|
||||
For instructions on how to document classes, exceptions, etc please consult the [Sphinx Napoleon Extension Guide][#7].
|
||||
|
||||
### Adding New Python Modules
|
||||
|
||||
When adding a new python binding module to the core of kaolin, a basic .rst will be automatically generated in docs/module/ when running ```make html``` (the automatic generation is not supported yet on windows).
|
||||
|
||||
The resulting .rst file will look like:
|
||||
```
|
||||
.. _moduleName:
|
||||
|
||||
<moduleName>
|
||||
============
|
||||
|
||||
.. automodule:: <moduleName>
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
```
|
||||
|
||||
If you want the .rst to not be generated, you must add the corresponding python path in [this list][#8].
|
||||
|
||||
To add more contents such as an introduction the .rst have to modified following RestructuredText syntax.
|
||||
|
||||
[#1]: https://www.python.org/dev/peps/pep-0257/
|
||||
[#2]: https://www.python.org/dev/peps/pep-0287/
|
||||
[#3]: https://devguide.python.org/documenting/
|
||||
[#4]: https://docs.python.org/3/library/typing.html
|
||||
[#5]: http://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
|
||||
[#6]: https://pybind11.readthedocs.io/en/stable/basics.html?highlight=py%3A%3Aarg#keyword-arguments
|
||||
[#7]: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html
|
||||
[#8]: https://github.com/NVIDIAGameWorks/kaolin/tree/master/docs/kaolin_ext.py#L21
|
||||
76
docs/_templates/layout.html
vendored
Normal file
@@ -0,0 +1,76 @@
|
||||
|
||||
{% extends "!layout.html" %}
|
||||
{% block footer %} {{ super() }}
|
||||
<style>
|
||||
:root {
|
||||
--nvidia-color: #76B900;
|
||||
--dark-green: #008564;
|
||||
}
|
||||
|
||||
a, a:visited, a:active {
|
||||
color: var(--nvidia-color);
|
||||
}
|
||||
|
||||
/* Sidebar header (and topbar for mobile) */
|
||||
.wy-side-nav-search, .wy-nav-top {
|
||||
background-color: var(--nvidia-color);
|
||||
}
|
||||
/* Sidebar */
|
||||
.wy-menu-vertical header, .wy-menu-vertical p.caption {
|
||||
color: #DFDFDF;
|
||||
}
|
||||
|
||||
.rst-content .note, .rst-content .seealso, .rst-content .wy-alert-info.admonition, .rst-content .wy-alert-info.admonition-todo, .rst-content .wy-alert-info.attention, .rst-content .wy-alert-info.caution, .rst-content .wy-alert-info.danger, .rst-content .wy-alert-info.error, .rst-content .wy-alert-info.hint, .rst-content .wy-alert-info.important, .rst-content .wy-alert-info.tip, .rst-content .wy-alert-info.warning, .wy-alert.wy-alert-info {
|
||||
background: #eaefe0;
|
||||
}
|
||||
|
||||
.rst-content .note .admonition-title, .rst-content .note .wy-alert-title, .rst-content .seealso .admonition-title, .rst-content .seealso .wy-alert-title, .rst-content .wy-alert-info.admonition-todo .admonition-title, .rst-content .wy-alert-info.admonition-todo .wy-alert-title, .rst-content .wy-alert-info.admonition .admonition-title, .rst-content .wy-alert-info.admonition .wy-alert-title, .rst-content .wy-alert-info.attention .admonition-title, .rst-content .wy-alert-info.attention .wy-alert-title, .rst-content .wy-alert-info.caution .admonition-title, .rst-content .wy-alert-info.caution .wy-alert-title, .rst-content .wy-alert-info.danger .admonition-title, .rst-content .wy-alert-info.danger .wy-alert-title, .rst-content .wy-alert-info.error .admonition-title, .rst-content .wy-alert-info.error .wy-alert-title, .rst-content .wy-alert-info.hint .admonition-title, .rst-content .wy-alert-info.hint .wy-alert-title, .rst-content .wy-alert-info.important .admonition-title, .rst-content .wy-alert-info.important .wy-alert-title, .rst-content .wy-alert-info.tip .admonition-title, .rst-content .wy-alert-info.tip .wy-alert-title, .rst-content .wy-alert-info.warning .admonition-title, .rst-content .wy-alert-info.warning .wy-alert-title, .rst-content .wy-alert.wy-alert-info .admonition-title, .wy-alert.wy-alert-info .rst-content .admonition-title, .wy-alert.wy-alert-info .wy-alert-title {
|
||||
background: #b8d27c;
|
||||
}
|
||||
|
||||
html.writer-html4 .rst-content dl:not(.docutils) dl:not(.field-list)>dt, html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.glossary):not(.simple) dl:not(.field-list):not(.simple)>dt.sig {
|
||||
background-color: #eaefe0;
|
||||
border-left: 3px solid var(--nvidia-color);
|
||||
}
|
||||
|
||||
html.writer-html4 .rst-content dl:not(.docutils)>dt, html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.glossary):not(.simple)>dt.sig {
|
||||
background: #eaefe0;
|
||||
border-top: 3px solid var(--nvidia-color);
|
||||
}
|
||||
|
||||
html.writer-html4 .rst-content dl:not(.docutils)>dt, html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.glossary):not(.simple)>dt {
|
||||
color: var(--dark-green);
|
||||
}
|
||||
|
||||
.icon, .version, a.icon.icon-home {
|
||||
color: white;
|
||||
}
|
||||
|
||||
table.center-align-center-col td {
|
||||
text-align: center
|
||||
}
|
||||
|
||||
.rubric, p.rubric {
|
||||
margin-bottom: 15px;
|
||||
font-weight: 700;
|
||||
font-size: 120%;
|
||||
color: var(--dark-green);
|
||||
border-bottom: 1px solid var(--dark-green);
|
||||
}
|
||||
|
||||
</style>
|
||||
{% endblock %}
|
||||
|
||||
{% block menu %}
|
||||
<p class="caption">
|
||||
<span class="caption-text">Getting Started:</span>
|
||||
</p>
|
||||
|
||||
<ul>
|
||||
<li class="toctree-l1"><a href="{{ pathto('index') }}">Welcome</a></li>
|
||||
<li class="toctree-l1"><a href="{{ pathto('notes/installation') }}">Installation</a></li>
|
||||
<li class="toctree-l1"><a href="{{ pathto('notes/overview') }}">API Overview</a></li>
|
||||
</ul>
|
||||
|
||||
{{super()}}
|
||||
{% endblock %}
|
||||
78
docs/conf.py
Normal file
@@ -0,0 +1,78 @@
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file only contains a selection of the most common options. For a full
|
||||
# list see the documentation:
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html
|
||||
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(1, os.path.abspath('.'))
|
||||
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = 'Kaolin'
|
||||
copyright = '2020, NVIDIA'
|
||||
author = 'NVIDIA'
|
||||
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
'sphinx.ext.autodoc',
|
||||
'sphinx.ext.napoleon',
|
||||
'sphinx.ext.intersphinx',
|
||||
'sphinx.ext.todo',
|
||||
'kaolin_ext'
|
||||
]
|
||||
|
||||
todo_include_todos = True
|
||||
|
||||
autodoc_typehints = "description"
|
||||
|
||||
intersphinx_mapping = {
|
||||
'python': ("https://docs.python.org/3", None),
|
||||
'numpy': ('https://numpy.org/doc/stable/', None),
|
||||
'PyTorch': ('https://pytorch.org/docs/master/', None),
|
||||
}
|
||||
|
||||
master_doc = 'index'
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path.
|
||||
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', '*.so']
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
import sphinx_rtd_theme
|
||||
|
||||
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
|
||||
html_theme = "sphinx_rtd_theme"
|
||||
html_theme_options = {
|
||||
'collapse_navigation': True
|
||||
}
|
||||
|
||||
# html_theme = 'alabaster'
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = []
|
||||
BIN
docs/img/answer.png
Normal file
|
After Width: | Height: | Size: 38 KiB |
BIN
docs/img/clock.gif
Normal file
|
After Width: | Height: | Size: 2.9 MiB |
BIN
docs/img/dash3d_viz.jpg
Normal file
|
After Width: | Height: | Size: 120 KiB |
BIN
docs/img/flexicubes.png
Normal file
|
After Width: | Height: | Size: 262 KiB |
BIN
docs/img/koala.jpg
Normal file
|
After Width: | Height: | Size: 626 KiB |
BIN
docs/img/mesh_to_spc.png
Normal file
|
After Width: | Height: | Size: 20 KiB |
BIN
docs/img/module.png
Normal file
|
After Width: | Height: | Size: 96 KiB |
BIN
docs/img/octants.png
Normal file
|
After Width: | Height: | Size: 65 KiB |
BIN
docs/img/octree.png
Normal file
|
After Width: | Height: | Size: 140 KiB |
BIN
docs/img/ov_viz.jpg
Normal file
|
After Width: | Height: | Size: 663 KiB |
BIN
docs/img/spcTeapot.png
Normal file
|
After Width: | Height: | Size: 108 KiB |
BIN
docs/img/spc_points.png
Normal file
|
After Width: | Height: | Size: 56 KiB |
35
docs/index.rst
Normal file
@@ -0,0 +1,35 @@
|
||||
Welcome to Kaolin Library Documentation
|
||||
=======================================
|
||||
|
||||
.. image:: ../assets/kaolin.png
|
||||
|
||||
`NVIDIA Kaolin library <https://github.com/NVIDIAGameWorks/kaolin>`_ provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints, differentiable camera API, differentiable lighting with spherical harmonics and spherical gaussians, powerful quadtree acceleration structure called Structured Point Clouds, interactive 3D visualizer for jupyter notebooks, convenient batched mesh container and more.
|
||||
See :ref:`Installation <installation>`, :ref:`API Overview <overview>` and :ref:`Tutorials <tutorial_index>` to get started!
|
||||
|
||||
Note that Kaolin library is part of the larger `NVIDIA Kaolin effort <https://developer.nvidia.com/kaolin>`_ for 3D deep learning.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:titlesonly:
|
||||
:maxdepth: 1
|
||||
:caption: Tutorials:
|
||||
|
||||
notes/tutorial_index
|
||||
notes/checkpoints
|
||||
notes/diff_render
|
||||
notes/spc_summary
|
||||
notes/differentiable_camera
|
||||
|
||||
.. toctree::
|
||||
:titlesonly:
|
||||
:maxdepth: 1
|
||||
:caption: API Reference:
|
||||
|
||||
modules/kaolin.ops
|
||||
modules/kaolin.metrics
|
||||
modules/kaolin.io
|
||||
modules/kaolin.render
|
||||
modules/kaolin.rep
|
||||
modules/kaolin.utils
|
||||
modules/kaolin.visualize
|
||||
modules/kaolin.non_commercial
|
||||
92
docs/kaolin_ext.py
Normal file
@@ -0,0 +1,92 @@
|
||||
# Copyright (c) 2019-2021, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import os
|
||||
|
||||
KAOLIN_ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir)
|
||||
|
||||
def run_apidoc(_):
|
||||
# This is runnning sphinx-apidoc which is automatically generating
|
||||
# .rst files for each python file in kaolin
|
||||
# This won't override existing .rst files
|
||||
# Like kaolin.ops.rst where we added an introduction
|
||||
from sphinx.ext import apidoc
|
||||
# Those are files are excluded from parsing
|
||||
# Such as files where the functions are forwarded to the parent namespace
|
||||
EXCLUDE_PATHS = [
|
||||
str(os.path.join(KAOLIN_ROOT, path)) for path in [
|
||||
"setup.py",
|
||||
"**.so",
|
||||
"kaolin/version.py",
|
||||
"kaolin/experimental/",
|
||||
"kaolin/version.txt",
|
||||
"kaolin/io/usd/utils.py",
|
||||
"kaolin/io/usd/mesh.py",
|
||||
"kaolin/io/usd/voxelgrid.py",
|
||||
"kaolin/io/usd/pointcloud.py",
|
||||
"kaolin/ops/conversions/pointcloud.py",
|
||||
"kaolin/ops/conversions/sdf.py",
|
||||
"kaolin/ops/conversions/trianglemesh.py",
|
||||
"kaolin/ops/conversions/voxelgrid.py",
|
||||
"kaolin/ops/conversions/tetmesh.py",
|
||||
"kaolin/ops/mesh/check_sign.py",
|
||||
"kaolin/ops/mesh/mesh.py",
|
||||
"kaolin/ops/mesh/tetmesh.py",
|
||||
"kaolin/ops/mesh/trianglemesh.py",
|
||||
"kaolin/ops/spc/spc.py",
|
||||
"kaolin/ops/spc/convolution.py",
|
||||
"kaolin/ops/spc/points.py",
|
||||
"kaolin/ops/spc/uint8.py",
|
||||
"kaolin/render/lighting/sg.py",
|
||||
"kaolin/render/lighting/sh.py",
|
||||
"kaolin/render/mesh/deftet.py",
|
||||
"kaolin/render/mesh/dibr.py",
|
||||
"kaolin/render/mesh/rasterization.py",
|
||||
"kaolin/render/mesh/utils.py",
|
||||
"kaolin/render/spc/raytrace.py",
|
||||
"kaolin/rep/spc.py",
|
||||
"kaolin/visualize/timelapse.py",
|
||||
"kaolin/visualize/ipython.py",
|
||||
"kaolin/framework/*",
|
||||
"kaolin/render/camera/camera.py",
|
||||
"kaolin/render/camera/coordinates.py",
|
||||
"kaolin/render/camera/extrinsics_backends.py",
|
||||
"kaolin/render/camera/extrinsics.py",
|
||||
"kaolin/render/camera/intrinsics_ortho.py",
|
||||
"kaolin/render/camera/intrinsics_pinhole.py",
|
||||
"kaolin/render/camera/intrinsics.py",
|
||||
"kaolin/render/camera/legacy.py",
|
||||
"kaolin/non_commercial/flexicubes/",
|
||||
"kaolin/non_commercial/flexicubes/flexicubes.py",
|
||||
"kaolin/non_commercial/flexicubes/tables.py"
|
||||
|
||||
]
|
||||
]
|
||||
|
||||
DOCS_MODULE_PATH = os.path.join(KAOLIN_ROOT, "docs", "modules")
|
||||
|
||||
argv = [
|
||||
"-eT",
|
||||
"-d", "2",
|
||||
"--templatedir",
|
||||
DOCS_MODULE_PATH,
|
||||
"-o", DOCS_MODULE_PATH,
|
||||
os.path.join(KAOLIN_ROOT, "kaolin"),
|
||||
*EXCLUDE_PATHS
|
||||
]
|
||||
apidoc.main(argv)
|
||||
os.remove(os.path.join(DOCS_MODULE_PATH, 'kaolin.rst'))
|
||||
|
||||
def setup(app):
|
||||
app.connect("builder-inited", run_apidoc)
|
||||
49
docs/make.bat
Normal file
@@ -0,0 +1,49 @@
|
||||
@ECHO OFF
|
||||
|
||||
pushd %~dp0
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
set KAOLIN_ROOT=%~dp0..
|
||||
echo "%KAOLIN_ROOT%\docs\modules\"
|
||||
|
||||
REM not to be used by end users
|
||||
set EXCLUDE_PATHS="%KAOLIN_ROOT%\kaolin\packed\ %KAOLIN_ROOT%\kaolin\padded\ %KAOLIN_ROOT%\kaolin\unbatched\ \kaolin\*_cuda.*"
|
||||
REM Those files are unused since we already have index.rst and conf.py
|
||||
set EXCLUDE_GEN_RST=%KAOLIN_ROOT%\docs\modules\setup.rst %KAOLIN_ROOT%\docs\modules\kaolin.rst %KAOLIN_ROOT%\docs\modules\kaolin.version.rst
|
||||
|
||||
sphinx-apidoc -eT -d 2 --templatedir=%KAOLIN_ROOT%\docs\modules\ -o %KAOLIN_ROOT%\docs\modules\ %KAOLIN_ROOT% %EXCLUDE_PATHS%
|
||||
|
||||
echo %EXCLUDE_GEN_RST%
|
||||
del %EXCLUDE_GEN_RST%
|
||||
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set SOURCEDIR=.
|
||||
set BUILDDIR=_build
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
%SPHINXBUILD% >NUL 2>NUL
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.http://sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
goto end
|
||||
|
||||
:help
|
||||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
|
||||
:end
|
||||
popd
|
||||
32
docs/modules/kaolin.io.materials.rst
Normal file
@@ -0,0 +1,32 @@
|
||||
.. _kaolin.io.materials:
|
||||
|
||||
kaolin.io.materials
|
||||
===================
|
||||
|
||||
.. currentmodule:: kaolin.io.materials
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
Functions
|
||||
|
||||
===================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.io.materials
|
||||
:members:
|
||||
:exclude-members:
|
||||
MaterialError,
|
||||
MaterialLoadError,
|
||||
MaterialFileError,
|
||||
MaterialNotFoundError
|
||||
|
||||
Exceptions
|
||||
----------
|
||||
|
||||
.. autoclass:: MaterialError
|
||||
.. autoclass:: MaterialLoadError
|
||||
.. autoclass:: MaterialFileError
|
||||
.. autoclass:: MaterialNotFoundError
|
||||
33
docs/modules/kaolin.io.obj.rst
Normal file
@@ -0,0 +1,33 @@
|
||||
.. _kaolin.io.obj:
|
||||
|
||||
kaolin.io.obj
|
||||
=============
|
||||
|
||||
.. currentmodule:: kaolin.io.obj
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
.. automodule:: kaolin.io.obj
|
||||
:members:
|
||||
:exclude-members:
|
||||
return_type,
|
||||
ignore_error_handler,
|
||||
skip_error_handler,
|
||||
default_error_handler,
|
||||
create_missing_materials_error_handler,
|
||||
MaterialError,
|
||||
MaterialLoadError,
|
||||
MaterialFileError,
|
||||
MaterialNotFoundError
|
||||
|
||||
Error Handler
|
||||
-------------
|
||||
|
||||
.. autofunction:: ignore_error_handler
|
||||
.. autofunction:: skip_error_handler
|
||||
.. autofunction:: default_error_handler
|
||||
.. autofunction:: create_missing_materials_error_handler
|
||||
26
docs/modules/kaolin.io.rst
Normal file
@@ -0,0 +1,26 @@
|
||||
.. _kaolin.io:
|
||||
|
||||
kaolin.io
|
||||
=========
|
||||
|
||||
IO directory contains all the functionalities to interact with data files.
|
||||
|
||||
:ref:`obj module<kaolin.io.obj>` and :ref:`usd module <kaolin.io.usd>` contains importer to .obj and o, importers / exporters to .usd(a) files,
|
||||
:ref:`dataset module<kaolin.io.dataset>` contains helper features for caching data, and preprocessing whole datasets,
|
||||
and :ref:`materials module<kaolin.io.materials>` contains Materials definition that should be used throughout Kaolin.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
kaolin.io.dataset
|
||||
kaolin.io.materials
|
||||
kaolin.io.gltf
|
||||
kaolin.io.obj
|
||||
kaolin.io.off
|
||||
kaolin.io.render
|
||||
kaolin.io.shapenet
|
||||
kaolin.io.usd
|
||||
kaolin.io.modelnet
|
||||
kaolin.io.shrec
|
||||
kaolin.io.utils
|
||||
10
docs/modules/kaolin.io.shapenet.rst
Normal file
@@ -0,0 +1,10 @@
|
||||
kaolin.io.shapenet
|
||||
==================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.io.shapenet
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
38
docs/modules/kaolin.io.usd.rst
Normal file
@@ -0,0 +1,38 @@
|
||||
.. _kaolin.io.usd:
|
||||
|
||||
kaolin.io.usd
|
||||
=============
|
||||
|
||||
Universal Scene Description
|
||||
---------------------------
|
||||
|
||||
Universal Scene Description (USD) is an open-source 3D scene description file format developed by Pixar and designed to be versatile, extensible and interchangeable between different 3D tools.
|
||||
|
||||
Single models and animations as well as large organized scenes composed of any number of assets can be defined in USD, making it suitable for organizing entire datasets into interpretable,
|
||||
subsets based on tags, class or other metadata label.
|
||||
|
||||
Kaolin includes base I/O operations for USD and also leverages this format to export 3D checkpoints. Use kaolin.io.usd to read and write USD files (try :code:`tutorials/usd_kitcheset.py`),
|
||||
and :code:`kaolin.visualize.Timelapse` to export 3D checkpoints (try :code:`tutorials/visualize_main.py`).
|
||||
|
||||
As a first step to familiarizing yourself with USD, we suggest following this `tutorial <https://developer.nvidia.com/usd>`_.
|
||||
More tutorials and documentation can be found `here <https://graphics.pixar.com/usd/docs/Introduction-to-USD.html>`_.
|
||||
|
||||
|
||||
Viewing USD Files
|
||||
~~~~~~~~~~~~~~~~~
|
||||
USD files can be visualized with realtime pathtracing using the [Omniverse Kaolin App](https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/user_manual.html#training-visualizer).
|
||||
Alternatively, you may use Pixar's USDView which can be obtained by visiting
|
||||
`https://developer.nvidia.com/usd <https://developer.nvidia.com/usd>`_ and selecting the
|
||||
corresponding platform under *USD Pre-Built Libraries and Tools*.
|
||||
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
.. automodule:: kaolin.io.usd
|
||||
:members:
|
||||
:exclude-members:
|
||||
mesh_return_type
|
||||
20
docs/modules/kaolin.metrics.rst
Normal file
@@ -0,0 +1,20 @@
|
||||
.. _kaolin.metrics:
|
||||
|
||||
kaolin.metrics
|
||||
==============
|
||||
|
||||
Metrics are differentiable operators that can be used to compute loss or accuracy.
|
||||
|
||||
We currently provide an IoU for voxelgrid, sided distance based metrics such as chamfer distance,
|
||||
point_to_mesh_distance and other simple regularization such as uniform_laplacian_smoothing.
|
||||
For tetrahedral mesh, we support the equivolume and AMIPS losses.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
kaolin.metrics.pointcloud
|
||||
kaolin.metrics.render
|
||||
kaolin.metrics.trianglemesh
|
||||
kaolin.metrics.voxelgrid
|
||||
kaolin.metrics.tetmesh
|
||||
12
docs/modules/kaolin.metrics.tetmesh.rst
Normal file
@@ -0,0 +1,12 @@
|
||||
.. _kaolin.metrics.tetmesh:
|
||||
|
||||
kaolin.metrics.tetmesh
|
||||
======================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.metrics.tetmesh
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
23
docs/modules/kaolin.non_commercial.rst
Normal file
@@ -0,0 +1,23 @@
|
||||
.. _kaolin.non_commercial:
|
||||
|
||||
kaolin.non\_commercial
|
||||
======================
|
||||
|
||||
License
|
||||
-------
|
||||
|
||||
This submodule contains features under `NSCL license <https://github.com/NVIDIAGameWorks/kaolin/blob/master/LICENSE.NSCL>`_ restricted to non commercial usage for research and evaluation purposes.
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.non_commercial.FlexiCubes
|
||||
:members:
|
||||
:special-members: __call__
|
||||
|
||||
.. automodule:: kaolin.non_commercial
|
||||
:members:
|
||||
:undoc-members:
|
||||
:exclude-members: FlexiCubes
|
||||
:show-inheritance:
|
||||
|
||||
83
docs/modules/kaolin.ops.batch.rst
Normal file
@@ -0,0 +1,83 @@
|
||||
.. _kaolin.ops.batch:
|
||||
|
||||
kaolin.ops.batch
|
||||
================
|
||||
|
||||
.. _batching:
|
||||
|
||||
Batching
|
||||
--------
|
||||
|
||||
Batching data in 3D can be tricky due to the heterogeneous sizes.
|
||||
|
||||
For instance, point clouds can have different number of points, which means we can't always just concatenate the tensors on a batch axis.
|
||||
|
||||
Kaolin supports different batching strategies:
|
||||
|
||||
.. _exact:
|
||||
|
||||
Exact
|
||||
~~~~~
|
||||
|
||||
Exact batching is the logical representation for homogeneous data.
|
||||
|
||||
For instance, if you sample the same numbers of points from a batch of meshes, you would just have a single tensor of shape :math:`(\text{batch_size}, \text{number_of_points}, 3)`.
|
||||
|
||||
.. _padded:
|
||||
|
||||
Padded
|
||||
~~~~~~
|
||||
|
||||
Heterogeneous tensors are padded to identical dimensions with a constant value so that they can be concatenated on a batch axis. This is similar to padding for the batching of image data of different shapes.
|
||||
|
||||
.. note::
|
||||
The last dimension must always be of the size of the element, e.g. 3 for 3D points (element of point clouds) or 1 for a grayscale pixel (element of grayscale textures).
|
||||
|
||||
For instance, for two textures :math:`T_0` and :math:`T_1` of shape :math:`(32, 32, 3)` and :math:`(64, 16, 3)`
|
||||
the batched tensor will be of shape :math:`(2, max(32, 64), max(32, 16), 3) = (2, 64, 32, 3)` and the padding value will be :math:`0`. :math:`T_0` will be padded on the 1st axis by :math:`32` while :math:`T_1` will be padded on the 2nd axis by :math:`16`.
|
||||
|
||||
You can also enforce a specific maximum shape (if you want to have a fix memory consumption or use optimization like cudnn algorithm selection).
|
||||
|
||||
For instance, you can force :math:`T_0` and :math:`T_1` to be batched with a maximum shape of :math:`(128, 128)`, the batched tensor will be of shape :math:`(2, 128, 128, 3)`, :math:`T_0` will be padded on the 1st axis and 2nd axis by 96 and :math:`T_1` will be padded on the 1st axis by :math:`64` and on the 2nd axis by :math:`112`.
|
||||
|
||||
For more information on how to do padded batching check :func:`kaolin.ops.batch.list_to_padded`
|
||||
|
||||
Related attributes:
|
||||
...................
|
||||
.. _padded_shape_per_tensor:
|
||||
|
||||
* :attr:`shape_per_tensor`: 2D :class:`torch.LongTensor` stores the shape of each sub-tensor except the last dimension in the padded tensor. E.g., in the example above :attr:`shape_per_tensor` would be ``torch.LongTensor([[32, 32], [64, 16]])``. Refer to :func:`kaolin.ops.batch.get_shape_per_tensor` for more information.
|
||||
|
||||
.. _packed:
|
||||
|
||||
Packed
|
||||
~~~~~~
|
||||
|
||||
Heterogeneous tensors are reshaped to 2D :math:`(-1, \text{last_dimension})` and concatenated on the first axis. This is similar to packed sentences in NLP.
|
||||
|
||||
.. note::
|
||||
The last dimension must always be of the size of the element, e.g. 3 for 3D points (element of point clouds) or 1 for a grayscale pixel (element of grayscale textures).
|
||||
|
||||
For instance, for two textures :math:`T_0` and :math:`T_1` of shape :math:`(32, 32, 3)` and :math:`(64, 16, 3)`
|
||||
The batched tensor will be of shape :math:`(32 * 32 + 64 * 16, 3)`. :math:`T_0` will be reshaped to :math:`(32 * 32, 3)` and :math:`T_1` will be reshaped :math:`(64 * 16, 3)`, before being concatenated on the first axis.
|
||||
|
||||
For more information on how to do padded batching check :func:`kaolin.ops.batch.list_to_packed`
|
||||
|
||||
Related attributes:
|
||||
...................
|
||||
.. _packed_shape_per_tensor:
|
||||
|
||||
* :attr:`shape_per_tensor`: 2D :class:`torch.LongTensor` stores the shape of each sub-tensor except the last dimension in the padded tensor. E.g., in the example above :attr:`shape_per_tensor` would be ``torch.LongTensor([[32, 32], [64, 16]])``. Refer to :func:`kaolin.ops.batch.get_shape_per_tensor` for more information.
|
||||
|
||||
.. _packed_first_idx:
|
||||
|
||||
* :attr:`first_idx`: 1D :class:`torch.LongTensor` stores the first index of each subtensor and the last index + 1 on the first axis in the packed tensor. E.g., in the example above :attr:`first_idx` would be ``torch.LongTensor([0, 1024, 2048])``. This attribute are used for delimiting each subtensor into the packed tensor, for instance, to slice or index. Refer to :func:`kaolin.ops.batch.get_first_idx` for more information.
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.ops.batch
|
||||
:platform: Windows-x86_64, Linux-x86_64
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
12
docs/modules/kaolin.ops.conversions.rst
Normal file
@@ -0,0 +1,12 @@
|
||||
.. _kaolin.ops.conversions:
|
||||
|
||||
kaolin.ops.conversions
|
||||
======================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.ops.conversions
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
38
docs/modules/kaolin.ops.mesh.rst
Normal file
@@ -0,0 +1,38 @@
|
||||
.. _kaolin.ops.mesh:
|
||||
|
||||
kaolin.ops.mesh
|
||||
***********************
|
||||
|
||||
A mesh is a 3D object representation consisting of a collection of vertices and polygons.
|
||||
|
||||
Triangular meshes
|
||||
==================
|
||||
|
||||
Triangular meshes comprise of a set of triangles that are connected by their common edges or corners. In Kaolin, they are usually represented as a set of two tensors:
|
||||
|
||||
* ``vertices``: A :class:`torch.Tensor`, of shape :math:`(\text{batch_size}, \text{num_vertices}, 3)`, contains the vertices coordinates.
|
||||
|
||||
* ``faces``: A :class:`torch.LongTensor`, of shape :math:`(\text{batch_size}, \text{num_faces}, 3)`, contains the mesh topology, by listing the vertices index for each face.
|
||||
|
||||
Both tensors can be combined using :func:`kaolin.ops.mesh.index_vertices_by_faces`, to form ``face_vertices``, of shape :math:`(\text{batch_size}, \text{num_faces}, 3, 3)`, listing the vertices coordinate for each face.
|
||||
|
||||
|
||||
Tetrahedral meshes
|
||||
==================
|
||||
|
||||
A tetrahedron or triangular pyramid is a polyhedron composed of four triangular faces, six straight edges, and four vertex corners. Tetrahedral meshes inside Kaolin are composed of two tensors:
|
||||
|
||||
* ``vertices``: A :class:`torch.Tensor`, of shape :math:`(\text{batch_size}, \text{num_vertices}, 3)`, contains the vertices coordinates.
|
||||
|
||||
* ``tet``: A :class:`torch.LongTensor`, of shape :math:`(\text{batch_size}, \text{num_tet}, 4)`, contains the tetrahedral mesh topology, by listing the vertices index for each tetrahedron.
|
||||
|
||||
Both tensors can be combined, to form ``tet_vertices``, of shape :math:`(\text{batch_size}, \text{num_tet}, 4, 3)`, listing the tetrahedrons vertices coordinates for each face.
|
||||
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.ops.mesh
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
23
docs/modules/kaolin.ops.rst
Normal file
@@ -0,0 +1,23 @@
|
||||
.. _kaolin.ops:
|
||||
|
||||
kaolin.ops
|
||||
==========
|
||||
|
||||
Operators are primitive processing functions for batched 3D models (:ref:`meshes<kaolin.ops.mesh>`, :ref:`voxelgrids<kaolin.ops.voxelgrid>` and point clouds).
|
||||
|
||||
Tensor batching operators are in :ref:`kaolin.ops.batch`, conversions of 3D models between different representations are in :ref:`kaolin.ops.conversions`.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
kaolin.ops.batch
|
||||
kaolin.ops.coords
|
||||
kaolin.ops.conversions
|
||||
kaolin.ops.pointcloud
|
||||
kaolin.ops.gcn
|
||||
kaolin.ops.mesh
|
||||
kaolin.ops.random
|
||||
kaolin.ops.reduction
|
||||
kaolin.ops.spc
|
||||
kaolin.ops.voxelgrid
|
||||
12
docs/modules/kaolin.ops.spc.rst
Normal file
@@ -0,0 +1,12 @@
|
||||
.. _kaolin.ops.spc:
|
||||
|
||||
kaolin.ops.spc
|
||||
##############
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.ops.spc
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
15
docs/modules/kaolin.render.camera.camera.rst
Normal file
@@ -0,0 +1,15 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.render.camera.Camera:
|
||||
|
||||
kaolin.render.camera.Camera
|
||||
===========================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.render.camera.Camera
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
14
docs/modules/kaolin.render.camera.camera_extrinsics.rst
Normal file
@@ -0,0 +1,14 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.render.camera.CameraExtrinsics:
|
||||
|
||||
kaolin.render.camera.CameraExtrinsics
|
||||
=====================================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.render.camera.CameraExtrinsics
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
14
docs/modules/kaolin.render.camera.camera_intrinsics.rst
Normal file
@@ -0,0 +1,14 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.render.camera.CameraIntrinsics:
|
||||
|
||||
kaolin.render.camera.CameraIntrinsics
|
||||
=====================================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.render.camera.CameraIntrinsics
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
14
docs/modules/kaolin.render.camera.extrinsics_rep.rst
Normal file
@@ -0,0 +1,14 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.render.camera.ExtrinsicsRep:
|
||||
|
||||
kaolin.render.camera.ExtrinsicsRep
|
||||
==================================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.render.camera.ExtrinsicsRep
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
@@ -0,0 +1,14 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.render.camera.OrthographicIntrinsics:
|
||||
|
||||
kaolin.render.camera.OrthographicIntrinsics
|
||||
===========================================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.render.camera.OrthographicIntrinsics
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
14
docs/modules/kaolin.render.camera.pinhole_intrinsics.rst
Normal file
@@ -0,0 +1,14 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.render.camera.PinholeIntrinsics:
|
||||
|
||||
kaolin.render.camera.PinholeIntrinsics
|
||||
======================================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.render.camera.PinholeIntrinsics
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
36
docs/modules/kaolin.render.camera.rst
Normal file
@@ -0,0 +1,36 @@
|
||||
.. _kaolin.render.camera:
|
||||
|
||||
kaolin.render.camera
|
||||
====================
|
||||
|
||||
Kaolin provides extensive camera API. For an overview, see the :ref:`Camera class docs <kaolin.render.camera.Camera>`.
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
Classes
|
||||
^^^^^^^
|
||||
|
||||
* :ref:`Camera <kaolin.render.camera.Camera>`
|
||||
* :ref:`CameraExtrinsics <kaolin.render.camera.CameraExtrinsics>`
|
||||
* :ref:`CameraIntrinsics <kaolin.render.camera.CameraIntrinsics>`
|
||||
* :ref:`PinholeIntrinsics <kaolin.render.camera.PinholeIntrinsics>`
|
||||
* :ref:`OrthographicIntrinsics <kaolin.render.camera.OrthographicIntrinsics>`
|
||||
* :ref:`ExtrinsicsRep <kaolin.render.camera.ExtrinsicsRep>`
|
||||
|
||||
Functions
|
||||
^^^^^^^^^
|
||||
|
||||
.. automodule:: kaolin.render.camera
|
||||
:members:
|
||||
:exclude-members:
|
||||
Camera,
|
||||
CameraExtrinsics,
|
||||
CameraIntrinsics,
|
||||
PinholeIntrinsics,
|
||||
OrthographicIntrinsics,
|
||||
ExtrinsicsRep
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
|
||||
13
docs/modules/kaolin.render.rst
Normal file
@@ -0,0 +1,13 @@
|
||||
.. _kaolin.render:
|
||||
|
||||
kaolin.render
|
||||
=============
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
kaolin.render.camera
|
||||
kaolin.render.lighting
|
||||
kaolin.render.mesh
|
||||
kaolin.render.spc
|
||||
12
docs/modules/kaolin.render.spc.rst
Normal file
@@ -0,0 +1,12 @@
|
||||
.. _kaolin.render.spc:
|
||||
|
||||
kaolin.render.spc
|
||||
=================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.render.spc
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
27
docs/modules/kaolin.rep.rst
Normal file
@@ -0,0 +1,27 @@
|
||||
.. _kaolin.rep:
|
||||
|
||||
kaolin.rep
|
||||
==========
|
||||
|
||||
This module includes higher-level Kaolin classes ("representations").
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
Classes
|
||||
^^^^^^^
|
||||
|
||||
* :ref:`SurfaceMesh <kaolin.rep.SurfaceMesh>`
|
||||
* :ref:`Spc <kaolin.rep.Spc>`
|
||||
|
||||
Other
|
||||
^^^^^^^^^
|
||||
|
||||
.. automodule:: kaolin.rep
|
||||
:members:
|
||||
:exclude-members:
|
||||
SurfaceMesh,
|
||||
Spc
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
14
docs/modules/kaolin.rep.spc.rst
Normal file
@@ -0,0 +1,14 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.rep.Spc:
|
||||
|
||||
kaolin.rep.Spc
|
||||
===========================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. autoclass:: kaolin.rep.Spc
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
146
docs/modules/kaolin.rep.surface_mesh.rst
Normal file
@@ -0,0 +1,146 @@
|
||||
:orphan:
|
||||
|
||||
.. _kaolin.rep.SurfaceMesh:
|
||||
|
||||
SurfaceMesh
|
||||
===========================
|
||||
|
||||
Tutorial
|
||||
--------
|
||||
|
||||
For a walk-through of :class:`kaolin.rep.SurfaceMesh` features,
|
||||
see `working_with_meshes.ipynb <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/working_with_meshes.ipynb>`_.
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
* :ref:`Overview <rubric mesh overview>`
|
||||
* :ref:`Supported Attributes <rubric mesh attributes>`
|
||||
* :ref:`Batching <rubric mesh batching>`
|
||||
* :ref:`Attribute Access and Auto-Computability <rubric mesh attribute access>`
|
||||
* :ref:`Inspecting and Copying <rubric mesh inspecting>`
|
||||
* :ref:`Tensor Operations <rubric mesh tensor ops>`
|
||||
|
||||
.. autoclass:: kaolin.rep.SurfaceMesh
|
||||
:members:
|
||||
:undoc-members:
|
||||
:member-order: bysource
|
||||
:exclude-members: Batching, attribute_info_string, set_batching, to_batched, getattr_batched, cat,
|
||||
vertices, face_vertices, normals, face_normals, vertex_normals, uvs, face_uvs, faces, face_normals_idx, face_uvs_idx,
|
||||
material_assignments, materials, cuda, cpu, to, float_tensors_to, detach, get_attributes, has_attribute, has_or_can_compute_attribute,
|
||||
probably_can_compute_attribute, get_attribute, get_or_compute_attribute, check_sanity, to_string, as_dict, describe_attribute,
|
||||
unset_attributes_return_none, allow_auto_compute, batching, convert_attribute_batching
|
||||
|
||||
|
||||
.. _rubric mesh batching:
|
||||
|
||||
.. rubric:: Supported Batching Strategies
|
||||
|
||||
``SurfaceMesh`` can be instantiated with any of the following batching
|
||||
strategies, and supports conversions between batching strategies. Current
|
||||
batching strategy of a ``mesh`` object can be read from ``mesh.batching`` or
|
||||
by running ``print(mesh)``.
|
||||
|
||||
For example::
|
||||
|
||||
mesh = kaolin.io.obj.load_mesh(path)
|
||||
print(mesh)
|
||||
mesh.to_batched()
|
||||
print(mesh)
|
||||
|
||||
.. autoclass:: kaolin.rep.SurfaceMesh.Batching
|
||||
:members:
|
||||
|
||||
.. automethod:: attribute_info_string
|
||||
.. automethod:: check_sanity
|
||||
.. automethod:: set_batching
|
||||
.. automethod:: to_batched
|
||||
.. automethod:: getattr_batched
|
||||
.. automethod:: cat
|
||||
.. automethod:: convert_attribute_batching
|
||||
|
||||
.. _rubric mesh attribute access:
|
||||
|
||||
.. rubric:: Attribute Access
|
||||
|
||||
By default, ``SurfaceMesh`` will attempt to auto-compute missing attributes
|
||||
on access. These attributes will be cached, unless their ancestors have
|
||||
``requires_grad == True``. This behavior of the ``mesh`` object can be changed
|
||||
at construction time (``allow_auto_compute=False``) or by setting
|
||||
``mesh.allow_auto_compute`` later. In addition to this convenience API,
|
||||
explicit methods for attribute access are also supported.
|
||||
|
||||
For example, using **convenience API**::
|
||||
|
||||
# Caching is enabled by default
|
||||
mesh = kaolin.io.obj.load_mesh(path, with_normals=False)
|
||||
print(mesh)
|
||||
print(mesh.has_attribute('face_normals')) # False
|
||||
fnorm = mesh.face_normals # Auto-computed
|
||||
print(mesh.has_attribute('face_normals')) # True (cached)
|
||||
|
||||
# Caching is disabled when gradients need to flow
|
||||
mesh = kaolin.io.obj.load_mesh(path, with_normals=False)
|
||||
mesh.vertices.requires_grad = True # causes caching to be off
|
||||
print(mesh.has_attribute('face_normals')) # False
|
||||
fnorm = mesh.face_normals # Auto-computed
|
||||
print(mesh.has_attribute('face_normals')) # False (caching disabled)
|
||||
|
||||
|
||||
For example, using **explicit API**::
|
||||
|
||||
mesh = kaolin.io.obj.load_mesh(path, with_normals=False)
|
||||
print(mesh.has_attribute('face_normals')) # False
|
||||
fnorm = mesh.get_or_compute_attribute('face_normals', should_cache=False)
|
||||
print(mesh.has_attribute('face_normals')) # False
|
||||
|
||||
|
||||
.. automethod:: get_attributes
|
||||
.. automethod:: has_attribute
|
||||
.. automethod:: has_or_can_compute_attribute
|
||||
.. automethod:: probably_can_compute_attribute
|
||||
.. automethod:: get_attribute
|
||||
.. automethod:: get_or_compute_attribute
|
||||
|
||||
.. _rubric mesh inspecting:
|
||||
|
||||
.. rubric:: Inspecting and Copying Meshes
|
||||
|
||||
To make it easier to work with, ``SurfaceMesh`` supports detailed print
|
||||
statements, as well as ``len()``, ``copy()``, ``deepcopy()`` and can be converted
|
||||
to a dictionary.
|
||||
|
||||
Supported operations::
|
||||
|
||||
import copy
|
||||
mesh_copy = copy.copy(mesh)
|
||||
mesh_copy = copy.deepcopy(mesh)
|
||||
batch_size = len(mesh)
|
||||
|
||||
# Print default attributes
|
||||
print(mesh)
|
||||
|
||||
# Print more detailed attributes
|
||||
print(mesh.to_string(detailed=True, print_stats=True))
|
||||
|
||||
# Print specific attribute
|
||||
print(mesh.describe_attribute('vertices'))
|
||||
|
||||
.. automethod:: to_string
|
||||
.. automethod:: describe_attribute
|
||||
.. automethod:: as_dict
|
||||
|
||||
.. _rubric mesh tensor ops:
|
||||
|
||||
.. rubric:: Tensor Operations
|
||||
|
||||
Convenience operations for device and type conversions of some or all member
|
||||
tensors.
|
||||
|
||||
.. automethod:: cuda
|
||||
.. automethod:: cpu
|
||||
.. automethod:: to
|
||||
.. automethod:: float_tensors_to
|
||||
.. automethod:: detach
|
||||
|
||||
.. rubric:: Other
|
||||
10
docs/modules/kaolin.utils.rst
Normal file
@@ -0,0 +1,10 @@
|
||||
.. _kaolin.utils:
|
||||
|
||||
kaolin.utils
|
||||
============
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
kaolin.utils.testing
|
||||
14
docs/modules/kaolin.visualize.rst
Normal file
@@ -0,0 +1,14 @@
|
||||
.. _kaolin.visualize:
|
||||
|
||||
kaolin.visualize
|
||||
================
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: kaolin.visualize
|
||||
:members:
|
||||
:inherited-members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
|
||||
14
docs/modules/module.rst_t
Normal file
@@ -0,0 +1,14 @@
|
||||
{%- if show_headings -%}
|
||||
.. _{{ basename }}:
|
||||
|
||||
{{ basename | e | heading }}
|
||||
{%- endif %}
|
||||
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: {{ qualname }}
|
||||
{%- for option in automodule_options %}
|
||||
:{{ option }}:
|
||||
{%- endfor %}
|
||||
|
||||
34
docs/modules/package.rst_t
Normal file
@@ -0,0 +1,34 @@
|
||||
{%- macro automodule(modname, options) -%}
|
||||
API
|
||||
---
|
||||
|
||||
.. automodule:: {{ modname }}
|
||||
{%- for option in options %}
|
||||
:{{ option }}:
|
||||
{%- endfor %}
|
||||
{%- endmacro %}
|
||||
|
||||
{%- macro unfoldtree(docnames) -%}
|
||||
{%- for docname in docnames %}
|
||||
{{ docname }}
|
||||
{%- endfor %}
|
||||
{%- endmacro -%}
|
||||
|
||||
{%- if is_namespace %}
|
||||
{{- [pkgname, "namespace"] | join(" ") | e | heading }}
|
||||
{% else -%}
|
||||
.. _{{ pkgname }}:
|
||||
|
||||
{{ pkgname | e | heading }}
|
||||
{%- endif -%}
|
||||
|
||||
{%- if subpackages or submodules %}
|
||||
.. toctree::
|
||||
:maxdepth: {{ maxdepth }}
|
||||
:titlesonly:
|
||||
{% endif -%}
|
||||
{{ unfoldtree(subpackages + submodules) }}
|
||||
|
||||
{% if not is_namespace -%}
|
||||
{{ automodule(pkgname, automodule_options) }}
|
||||
{% endif %}
|
||||
8
docs/modules/toc.rst_t
Normal file
@@ -0,0 +1,8 @@
|
||||
{{ header | heading }}
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: {{ maxdepth }}
|
||||
{% for docname in docnames %}
|
||||
{{ docname }}
|
||||
{%- endfor %}
|
||||
|
||||
149
docs/notes/checkpoints.rst
Normal file
@@ -0,0 +1,149 @@
|
||||
.. _3d_viz:
|
||||
|
||||
3D Checkpoint Visualization
|
||||
===========================
|
||||
|
||||
.. image:: ../img/koala.jpg
|
||||
|
||||
Visualizing 3D inputs and outputs of your model during training is an
|
||||
essential diagnostic tool. Kaolin provides a :ref:`simple API to checkpoint<writing checkpoints>` **batches of meshes, pointclouds and voxelgrids**, as well as **colors and
|
||||
textures**, saving them in :ref:`the USD format<file format>`. These checkpoints can then be visualized locally using :ref:`Kaolin Omniverse App<ov app>` or by launching :ref:`Kaolin Dash3D<dash 3d>` on the commandline, allowing remote visualization through a web browser.
|
||||
|
||||
.. _writing checkpoints:
|
||||
|
||||
Writing Checkpoints:
|
||||
--------------------
|
||||
|
||||
In a common scenario, model performance is visualized for a
|
||||
small evaluation batch. Bootstrap 3D checkpoints in your python training
|
||||
code by configuring a :class:`~kaolin.visualize.Timelapse` object::
|
||||
|
||||
import kaolin
|
||||
timelapse = kaolin.visualize.Timelapse(viz_log_dir)
|
||||
|
||||
The ``viz_log_dir`` is the directory where checkpoints will be saved. Timelapse will create files and subdirectories under this path, so providing
|
||||
a dedicated ``viz_log_dir`` separate from your other logs and configs will help keep things clean. The :class:`~kaolin.visualize.Timelapse` API supports point clouds,
|
||||
voxel grids and meshes, as well as colors and textures.
|
||||
|
||||
Saving Fixed Data
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
To save any iteration-independent data,
|
||||
call ``timelapse`` before your training loop
|
||||
without providing an ``iteration`` parameter, e.g.::
|
||||
|
||||
timelapse.add_mesh_batch(category='ground_truth',
|
||||
faces_list=face_list,
|
||||
vertices_list=gt_vert_list)
|
||||
timelapse.add_pointcloud_batch(category='input',
|
||||
pointcloud_list=input_pt_clouds)
|
||||
|
||||
The ``category`` identifies the meaning of the data. In this toy example,
|
||||
the model learns to turn the ``'input'`` pointcloud into the ``'output'`` mesh. Both the ``'ground_truth'`` mesh and the ``'input'`` pointcloud batches are only saved once for easy visual comparison.
|
||||
|
||||
Saving Time-varying Data
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
To checkpoint time-varying data during training, simply call :meth:`~kaolin.visualize.Timelapse.add_mesh_batch`, :meth:`~kaolin.visualize.Timelapse.add_pointcloud_batch` or :meth:`~kaolin.visualize.Timelapse.add_voxelgrid_batch`, for example::
|
||||
|
||||
if iteration % checkpoint_interval == 0:
|
||||
timelapse.add_mesh_batch(category='output',
|
||||
iteration=iteration,
|
||||
faces_list=face_list,
|
||||
vertices_list=out_vert_list)
|
||||
|
||||
.. Tip::
|
||||
For any data type, only time-varying data needs to be saved at every iteration. E.g., if your output mesh topology is fixed, only save ``faces_list`` once, and then call ``add_mesh_batch`` with only the predicted ``vertices_list``. This will cut down your checkpoint size.
|
||||
|
||||
Saving Colors and Appearance
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
We are working on adding support for colors and semantic ids to
|
||||
point cloud and voxel grid checkpoints. Mesh API supports multiple time-varying materials
|
||||
by specifying a :class:`kaolin.io.PBRMaterial`. For an example
|
||||
of using materials, see
|
||||
`test_timelapse.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/tests/python/kaolin/visualize/test_timelapse.py>`_.
|
||||
|
||||
Sample Code
|
||||
^^^^^^^^^^^
|
||||
We provide a `script <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/visualize_main.py>`_ that writes mock checkpoints, which can be run as follows::
|
||||
|
||||
python examples/tutorial/visualize_main.py \
|
||||
--test_objs=path/to/object1.obj,path/to/object2.obj \
|
||||
--output_dir=path/to/logdir
|
||||
|
||||
In addition, see :ref:`diff_render` tutorial.
|
||||
|
||||
.. _file format:
|
||||
|
||||
Understanding the File Format:
|
||||
------------------------------
|
||||
|
||||
Kaolin :class:`~kaolin.visualize.Timelapse` writes checkpoints using Universal Scene Descriptor (USD) file format (`Documentation <https://graphics.pixar.com/usd/docs/index.html>`_), developed with wide support for use cases in visual effects, including time-varying data. This allows reducing redundancy in written
|
||||
data across time.
|
||||
|
||||
After checkpointing with :class:`~kaolin.visualize.Timelapse`, the input ``viz_log_dir`` will contain
|
||||
a similar file structure::
|
||||
|
||||
ground_truth/mesh_0.usd
|
||||
ground_truth/mesh_1.usd
|
||||
ground_truth/mesh_...
|
||||
ground_truth/textures
|
||||
input/pointcloud_0.usd
|
||||
input/pointcloud_1.usd
|
||||
input/pointcloud_...
|
||||
output/mesh_0.usd
|
||||
output/mesh_1.usd
|
||||
output/mesh_...
|
||||
output/pointcloud_0.usd
|
||||
output/pointcloud_1.usd
|
||||
output/pointcloud_...
|
||||
output/textures
|
||||
|
||||
Here, the root folder names correspond to the ``category`` parameter
|
||||
provided to :class:`~kaolin.visualize.Timelapse` functions. Each element
|
||||
of the batch of every type is saved in its own numbered ``.usd`` file. Each USD file can be viewed on its
|
||||
own using any USD viewer, such as `NVIDIA Omniverse View <https://www.nvidia.com/en-us/omniverse/apps/view/>`_, or the whole log directory can be visualized
|
||||
using the tools below.
|
||||
|
||||
.. Caution::
|
||||
Timelapse is designed to only save one visualization batch for every category and type. Saving multiple batches without interleaving the data can be accomplished by creating custom categories.
|
||||
|
||||
.. _ov app:
|
||||
|
||||
Visualizing with Kaolin Omniverse App:
|
||||
--------------------------------------
|
||||
|
||||
.. image:: ../img/ov_viz.jpg
|
||||
|
||||
USD checkpoints can be visualized using a dedicated Omniverse Kaolin App `Training Visualizer <https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/user_manual.html#training-visualizer>`_.
|
||||
This extension provides full-featured support and high-fidelity rendering
|
||||
of all data types and materials that can be exported using :class:`~kaolin.visualize.Timelapse`, and allows creating custom visualization layouts and viewing meshes in multiple time-varying materials. `Download NVIDIA Omniverse <https://www.nvidia.com/en-us/omniverse/>`_ to get started!
|
||||
|
||||
.. _dash 3d:
|
||||
|
||||
Visualizing with Kaolin Dash3D:
|
||||
-------------------------------
|
||||
|
||||
.. image:: ../img/dash3d_viz.jpg
|
||||
|
||||
Omniverse app requires local access to a GPU and to the saved checkpoints, which is not always possible.
|
||||
We are also developing a lightweight ``kaolin-dash3d`` visualizer,
|
||||
which allows visualizing local and remote checkpoints without specialized
|
||||
hardware or applications. This tool is bundled with the latest
|
||||
builds as a command-line utility
|
||||
|
||||
To start Dash3D on the machine that stores the checkpoints, run::
|
||||
|
||||
kaolin-dash3d --logdir=$TIMELAPSE_DIR --port=8080
|
||||
|
||||
The ``logdir`` is the directory :class:`kaolin.visualize.Timelapse` was configured with. This command will launch a web server that will stream
|
||||
geometry to web clients. To connect, simply visit ``http://ip.of.machine:8080`` (or `localhost:8080 <http://localhost:8080/>`_ if connecting locally or with ssh port forwarding).
|
||||
|
||||
Try it now:
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
See Dash3D in action by running it on our test samples and visiting `localhost:8080 <http://localhost:8080/>`_::
|
||||
|
||||
kaolin-dash3d --logdir=$KAOLIN_ROOT/tests/samples/timelapse/notexture/ --port=8080
|
||||
|
||||
.. Caution:: Dash3d is still an experimental feature under active development. It only supports **triangle meshes** and **pointclouds** and cannot yet visualize colors, ids or textures. The web client was tested the most on `Google Chrome <https://www.google.com/chrome/>`_. We welcome your early feedback on our `github <https://github.com/NVIDIAGameWorks/kaolin/issues>`_!
|
||||
13
docs/notes/diff_render.rst
Normal file
@@ -0,0 +1,13 @@
|
||||
.. _diff_render:
|
||||
|
||||
Differentiable Rendering
|
||||
========================
|
||||
|
||||
.. image:: ../img/clock.gif
|
||||
|
||||
Differentiable rendering can be used to optimize the underlying 3D properties, like geometry and lighting, by backpropagating gradients from the loss in the image space. We provide an end-to-end tutorial for using the :mod:`kaolin.render.mesh` API in a Jupyter notebook:
|
||||
|
||||
`examples/tutorial/dibr_tutorial.ipynb <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/dibr_tutorial.ipynb>`_
|
||||
|
||||
In addition to the rendering API, the tutorial uses Omniverse Kaolin App `Data Generator <https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/user_manual.html#data-generator>`_ to create training data, :class:`kaolin.visualize.Timelapse` to write checkpoints, and
|
||||
Omniverse Kaolin App `Training Visualizer <https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/user_manual.html#training-visualizer>`_ to visualize them.
|
||||
237
docs/notes/differentiable_camera.rst
Normal file
@@ -0,0 +1,237 @@
|
||||
Differentiable Camera
|
||||
*********************
|
||||
|
||||
.. _differentiable_camera:
|
||||
|
||||
Camera class
|
||||
============
|
||||
|
||||
.. _camera_class:
|
||||
|
||||
:class:`kaolin.render.camera.Camera` is a one-stop class for all camera related differentiable / non-differentiable transformations.
|
||||
Camera objects are represented by *batched* instances of 2 submodules:
|
||||
|
||||
- :ref:`CameraExtrinsics <camera_extrinsics_class>`: The extrinsics properties of the camera (position, orientation).
|
||||
These are usually embedded in the view matrix, used to transform vertices from world space to camera space.
|
||||
- :ref:`CameraIntrinsics <camera_intrinsics_class>`: The intrinsics properties of the lens
|
||||
(such as field of view / focal length in the case of pinhole cameras).
|
||||
Intrinsics parameters vary between different lens type,
|
||||
and therefore multiple CameraIntrinsics subclasses exist,
|
||||
to support different types of cameras: pinhole / perspective, orthographic, fisheye, and so forth.
|
||||
For pinehole and orthographic lens, the intrinsics are embedded in a projection matrix.
|
||||
The intrinsics module can be used to transform vertices from camera space to Normalized Device Coordinates.
|
||||
|
||||
.. note::
|
||||
To avoid tedious invocation of camera functions through
|
||||
``camera.extrinsics.someop()`` and ``camera.intrinsics.someop()``, kaolin overrides the ``__get_attributes__``
|
||||
function to forward any function calls of ``camera.someop()`` to
|
||||
the appropriate extrinsics / intrinsics submodule.
|
||||
|
||||
The entire pipeline of transformations can be summarized as (ignoring homogeneous coordinates)::
|
||||
|
||||
World Space Camera View Space
|
||||
V ---CameraExtrinsics.transform()---> V' ---CameraIntrinsics.transform()---
|
||||
Shape~(B, 3) (view matrix) Shape~(B, 3) |
|
||||
|
|
||||
(linear lens: projection matrix) |
|
||||
+ homogeneus -> 3D |
|
||||
V
|
||||
Normalized Device Coordinates (NDC)
|
||||
Shape~(B, 3)
|
||||
When using view / projection matrices, conversion to homogeneous coordinates is required.
|
||||
Alternatively, the `transform()` function takes care of such projections under the hood when needed.
|
||||
|
||||
How to apply transformations with kaolin's Camera:
|
||||
1. Linear camera types, such as the commonly used pinhole camera,
|
||||
support the :func:`view_projection_matrix()` method.
|
||||
The returned matrix can be used to transform vertices through pytorch's matrix multiplication, or even be
|
||||
passed to shaders as a uniform.
|
||||
2. All Cameras are guaranteed to support a general :func:`transform()` function
|
||||
which maps coordinates from world space to Normalized Device Coordinates space.
|
||||
For some lens types which perform non linear transformations,
|
||||
the :func:`view_projection_matrix()` is non-defined.
|
||||
Therefore the camera transformation must be applied through
|
||||
a dedicated function. For linear cameras,
|
||||
:func:`transform()` may use matrices under the hood.
|
||||
3. Camera parameters may also be queried directly.
|
||||
This is useful when implementing camera params aware code such as ray tracers.
|
||||
How to control kaolin's Camera:
|
||||
- :class:`CameraExtrinsics`: is packed with useful methods for controlling the camera position and orientation:
|
||||
:func:`translate() <CameraExtrinsics.translate()>`,
|
||||
:func:`rotate() <CameraExtrinsics.rotate()>`,
|
||||
:func:`move_forward() <CameraExtrinsics.move_forward()>`,
|
||||
:func:`move_up() <CameraExtrinsics.move_up()>`,
|
||||
:func:`move_right() <CameraExtrinsics.move_right()>`,
|
||||
:func:`cam_pos() <CameraExtrinsics.cam_pos()>`,
|
||||
:func:`cam_up() <CameraExtrinsics.cam_up()>`,
|
||||
:func:`cam_forward() <CameraExtrinsics.cam_forward()>`,
|
||||
:func:`cam_up() <CameraExtrinsics.cam_up()>`.
|
||||
- :class:`CameraIntrinsics`: exposes a lens :func:`zoom() <CameraIntrinsics.zoom()>`
|
||||
operation. The exact functionality depends on the camera type.
|
||||
How to optimize the Camera parameters:
|
||||
- Both :class:`CameraExtrinsics`: and :class:`CameraIntrinsics` maintain
|
||||
:class:`torch.Tensor` buffers of parameters which support pytorch differentiable operations.
|
||||
- Setting ``camera.requires_grad_(True)`` will turn on the optimization mode.
|
||||
- The :func:`gradient_mask` function can be used to mask out gradients of specific Camera parameters.
|
||||
|
||||
.. note::
|
||||
:class:`CameraExtrinsics`: supports multiple representions of camera parameters
|
||||
(see: :func:`switch_backend <CameraExtrinsics.switch_backend()>`).
|
||||
Specific representations are better fit for optimization
|
||||
(e.g.: they maintain an orthogonal view matrix).
|
||||
Kaolin will automatically switch to using those representations when gradient flow is enabled
|
||||
For non-differentiable uses, the default representation may provide better
|
||||
speed and numerical accuracy.
|
||||
|
||||
Other useful camera properties:
|
||||
- Cameras follow pytorch in part, and support arbitrary ``dtype`` and ``device`` types through the
|
||||
:func:`to()`, :func:`cpu()`, :func:`cuda()`, :func:`half()`, :func:`float()`, :func:`double()`
|
||||
methods and :func:`dtype`, :func:`device` properties.
|
||||
- :class:`CameraExtrinsics`: and :class:`CameraIntrinsics`: individually support the :func:`requires_grad`
|
||||
property.
|
||||
- Cameras implement :func:`torch.allclose` for comparing camera parameters under controlled numerical accuracy.
|
||||
The operator ``==`` is reserved for comparison by ref.
|
||||
- Cameras support batching, either through construction, or through the :func:`cat()` method.
|
||||
|
||||
.. note::
|
||||
Since kaolin's cameras are batched, the view/projection matrices are of shapes :math:`(\text{num_cameras}, 4, 4)`,
|
||||
and some operations, such as :func:`transform()` may return values as shapes of :math:`(\text{num_cameras}, \text{num_vectors}, 3)`.
|
||||
|
||||
Concluding remarks on coordinate systems and other confusing conventions:
|
||||
- kaolin's Cameras assume column major matrices, for example, the inverse view matrix (cam2world) is defined as:
|
||||
|
||||
.. math::
|
||||
\begin{bmatrix}
|
||||
r1 & u1 & f1 & px \\
|
||||
r2 & u2 & f2 & py \\
|
||||
r3 & u3 & f3 & pz \\
|
||||
0 & 0 & 0 & 1
|
||||
\end{bmatrix}
|
||||
|
||||
This sometimes causes confusion as the view matrix (world2cam) uses a transposed 3x3 submatrix component,
|
||||
which despite this transposition is still column major (observed through the last `t` column):
|
||||
|
||||
.. math::
|
||||
\begin{bmatrix}
|
||||
r1 & r2 & r3 & tx \\
|
||||
u1 & u2 & u3 & ty \\
|
||||
f1 & f2 & f3 & tz \\
|
||||
0 & 0 & 0 & 1
|
||||
\end{bmatrix}
|
||||
|
||||
- kaolin's cameras do not assume any specific coordinate system for the camera axes. By default, the
|
||||
right handed cartesian coordinate system is used. Other coordinate systems are supported through
|
||||
:func:`change_coordinate_system() <CameraExtrinsics.change_coordinate_system()>`
|
||||
and the ``coordinates.py`` module::
|
||||
|
||||
Y
|
||||
^
|
||||
|
|
||||
|---------> X
|
||||
/
|
||||
Z - kaolin's NDC space is assumed to be left handed (depth goes inwards to the screen).
|
||||
|
||||
The default range of values is [-1, 1].
|
||||
|
||||
CameraExtrinsics class
|
||||
======================
|
||||
|
||||
.. _camera_extrinsics_class:
|
||||
|
||||
:class:`kaolin.render.camera.CameraExtrinsics` holds the extrinsics parameters of a camera: position and orientation in space.
|
||||
|
||||
This class maintains the view matrix of camera, used to transform points from world coordinates
|
||||
to camera / eye / view space coordinates.
|
||||
|
||||
This view matrix maintained by this class is column-major, and can be described by the 4x4 block matrix:
|
||||
|
||||
.. math::
|
||||
|
||||
\begin{bmatrix}
|
||||
R & t \\
|
||||
0 & 1
|
||||
\end{bmatrix}
|
||||
|
||||
where **R** is a 3x3 rotation matrix and **t** is a 3x1 translation vector for the orientation and position
|
||||
respectively.
|
||||
|
||||
This class is batched and may hold information from multiple cameras.
|
||||
|
||||
:class:`CameraExtrinsics` relies on a dynamic representation backend to manage the tradeoff between various choices
|
||||
such as speed, or support for differentiable rigid transformations.
|
||||
Parameters are stored as a single tensor of shape :math:`(\text{num_cameras}, K)`,
|
||||
where K is a representation specific number of parameters.
|
||||
Transformations and matrices returned by this class support differentiable torch operations,
|
||||
which in turn may update the extrinsic parameters of the camera::
|
||||
|
||||
convert_to_mat
|
||||
Backend ---- > Extrinsics
|
||||
Representation R View Matrix M
|
||||
Shape (num_cameras, K), Shape (num_cameras, 4, 4)
|
||||
< ----
|
||||
convert_from_mat
|
||||
|
||||
.. note::
|
||||
|
||||
Unless specified manually with :func:`switch_backend`,
|
||||
kaolin will choose the optimal representation backend depending on the status of ``requires_grad``.
|
||||
.. note::
|
||||
|
||||
Users should be aware, but not concerned about the conversion from internal representations to view matrices.
|
||||
kaolin performs these conversions where and if needed.
|
||||
|
||||
Supported backends:
|
||||
|
||||
- **"matrix_se3"**\: A flattened view matrix representation, containing the full information of
|
||||
special euclidean transformations (translations and rotations).
|
||||
This representation is quickly converted to a view matrix, but differentiable ops may cause
|
||||
the view matrix to learn an incorrect, non-orthogonal transformation.
|
||||
- **"matrix_6dof_rotation"**\: A compact representation with 6 degrees of freedom, ensuring the view matrix
|
||||
remains orthogonal under optimizations. The conversion to matrix requires a single Gram-Schmidt step.
|
||||
|
||||
.. seealso::
|
||||
|
||||
`On the Continuity of Rotation Representations in Neural Networks, Zhou et al. 2019
|
||||
<https://arxiv.org/abs/1812.07035>`_
|
||||
|
||||
Unless stated explicitly, the definition of the camera coordinate system used by this class is up to the
|
||||
choice of the user.
|
||||
Practitioners should be mindful of conventions when pairing the view matrix managed by this class with a projection
|
||||
matrix.
|
||||
|
||||
CameraIntrinsics class
|
||||
======================
|
||||
|
||||
.. _camera_intrinsics_class:
|
||||
|
||||
:class:`kaolin.render.camera.CameraIntrinsics` holds the intrinsics parameters of a camera:
|
||||
how it should project from camera space to normalized screen / clip space.
|
||||
|
||||
The instrinsics are determined by the camera type, meaning parameters may differ according to the lens structure.
|
||||
Typical computer graphics systems commonly assume the intrinsics of a pinhole camera (see: :class:`PinholeIntrinsics` class).
|
||||
One implication is that some camera types do not use a linear projection (i.e: Fisheye lens).
|
||||
|
||||
There are therefore numerous ways to use CameraIntrinsics subclasses:
|
||||
|
||||
1. Access intrinsics parameters directly.
|
||||
This may typically benefit use cases such as ray generators.
|
||||
2. The :func:`transform()` method is supported by all CameraIntrinsics subclasses,
|
||||
both linear and non-linear transformations, to project vectors from camera space to normalized screen space.
|
||||
This method is implemented using differential pytorch operations.
|
||||
3. Certain CameraIntrinsics subclasses which perform linear projections, may expose the transformation matrix
|
||||
via dedicated methods.
|
||||
For example, :class:`PinholeIntrinsics` exposes a :func:`projection_matrix()` method.
|
||||
This may typically be useful for rasterization based rendering pipelines (i.e: OpenGL vertex shaders).
|
||||
|
||||
This class is batched and may hold information from multiple cameras.
|
||||
Parameters are stored as a single tensor of shape :math:`(\text{num_cameras}, K)` where K is the number of
|
||||
intrinsic parameters.
|
||||
|
||||
currently there are two subclasses of intrinsics: :class:`kaolin.render.camera.OrthographicIntrinsics` and
|
||||
:class:`kaolin.render.camera.PinholeIntrinsics`.
|
||||
|
||||
API Documentation:
|
||||
------------------
|
||||
|
||||
* Check all the camera classes and functions at the :ref:`API documentation<kaolin.render.camera>`.
|
||||
|
||||
158
docs/notes/installation.rst
Normal file
@@ -0,0 +1,158 @@
|
||||
:orphan:
|
||||
|
||||
.. _installation:
|
||||
|
||||
Installation
|
||||
============
|
||||
|
||||
Most functions in Kaolin use PyTorch with custom high-performance code in C++ and CUDA. For this reason,
|
||||
full Kaolin functionality is only available for systems with an NVIDIA GPU, supporting CUDA. While it is possible to install
|
||||
Kaolin on other systems, only a fraction of operations will be available for a CPU-only install.
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
* Linux, Windows, or macOS (CPU-only)
|
||||
* Python >= 3.8, <= 3.10
|
||||
* `CUDA <https://developer.nvidia.com/cuda-toolkit>`_ >= 10.0 (with 'nvcc' installed) See `CUDA Toolkit Archive <https://developer.nvidia.com/cuda-toolkit-archive>`_ to install older version.
|
||||
* torch >= 1.8, <= 2.1.1
|
||||
|
||||
Quick Start (Linux, Windows)
|
||||
----------------------------
|
||||
| Make sure any of the supported CUDA and torch versions below are pre-installed.
|
||||
| The latest version of Kaolin can be installed with pip:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install kaolin==0.15.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-{TORCH_VER}_cu{CUDA_VER}.html
|
||||
|
||||
.. Note::
|
||||
Replace *TORCH_VER* and *CUDA_VER* with any of the compatible options below.
|
||||
|
||||
|
||||
.. rst-class:: center-align-center-col
|
||||
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch / CUDA** | **cu113** | **cu116** | **cu117** | **cu118** | **cu121** |
|
||||
+==================+===========+===========+===========+===========+===========+
|
||||
| **torch-2.1.1** | | | | ✓ | ✓ |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch-2.1.0** | | | | ✓ | ✓ |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch-2.0.1** | | | ✓ | ✓ | |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch-2.0.0** | | | ✓ | ✓ | |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch-1.13.1** | | ✓ | ✓ | | |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch-1.13.0** | | ✓ | ✓ | | |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch-1.12.1** | ✓ | ✓ | | | |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
| **torch-1.12.0** | ✓ | ✓ | | | |
|
||||
+------------------+-----------+-----------+-----------+-----------+-----------+
|
||||
|
||||
For example, to install kaolin for torch 1.12.1 and CUDA 11.3:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install kaolin==0.15.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-1.12.1_cu113.html
|
||||
|
||||
You can check https://nvidia-kaolin.s3.us-east-2.amazonaws.com/index.html to see all the wheels available.
|
||||
|
||||
Installation from source
|
||||
------------------------
|
||||
|
||||
.. Note::
|
||||
We recommend installing Kaolin into a virtual environment. For instance to create a new environment with `Anaconda <https://www.anaconda.com/>`_:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ conda create --name kaolin python=3.8
|
||||
$ conda activate kaolin
|
||||
|
||||
1. Clone Repository
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Clone and optionally check out an `official release <https://github.com/NVIDIAGameWorks/kaolin/tags>`_:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ git clone --recursive https://github.com/NVIDIAGameWorks/kaolin
|
||||
$ cd kaolin
|
||||
$ git checkout v0.15.0 # optional
|
||||
|
||||
2. Install dependencies
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
You can install the dependencies running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install -r tools/build_requirements.txt -r tools/viz_requirements.txt -r tools/requirements.txt
|
||||
|
||||
2. Test CUDA
|
||||
^^^^^^^^^^^^
|
||||
|
||||
You can verify that CUDA is properly installed at the desired version with nvcc by running the following:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ nvidia-smi
|
||||
$ nvcc --version
|
||||
|
||||
3. Install Pytorch
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Follow `official instructions <https://pytorch.org>`_ to install PyTorch of a supported version.
|
||||
Kaolin may be able to work with other PyTorch versions, but we only explicitly test within the version range 1.10.0 to 2.1.1.
|
||||
See below for overriding PyTorch version check during install.
|
||||
|
||||
Here is how to install the latest Pytorch version supported by Kaolin for cuda 11.8:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install torch==2.1.1 --extra-index-url https://download.pytorch.org/whl/cu118
|
||||
|
||||
|
||||
4. Optional Environment Variables
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
* If trying Kaolin with an unsupported PyTorch version, set: ``export IGNORE_TORCH_VER=1``
|
||||
* If using heterogeneous GPU setup, set the architectures for which to compile the CUDA code, e.g.: ``export TORCH_CUDA_ARCH_LIST="7.0 7.5"``
|
||||
* In some setups, there may be a conflict between cub available with cuda install > 11 and ``third_party/cub`` that kaolin includes as a submodule. If conflict occurs or cub is not found, set ``CUB_HOME`` to the cuda one, e.g. typically on Linux: ``export CUB_HOME=/usr/local/cuda-*/include/``
|
||||
|
||||
|
||||
5. Install Kaolin
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python setup.py develop
|
||||
|
||||
.. Note::
|
||||
Kaolin can be installed without GPU, however, CPU support is limited and many CUDA-only functions will be missing.
|
||||
|
||||
Testing your installation
|
||||
-------------------------
|
||||
|
||||
Run a quick test of your installation and version:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python -c "import kaolin; print(kaolin.__version__)"
|
||||
|
||||
Running tests
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
For an exhaustive check, install testing dependencies and run tests as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install -r tools/ci_requirements.txt
|
||||
$ export CI='true' # on Linux
|
||||
$ set CI='true' # on Windows
|
||||
$ pytest --import-mode=importlib -s tests/python/
|
||||
|
||||
.. Note::
|
||||
These tests rely on CUDA operations and will fail if you installed on CPU only, where not all functionality is available.
|
||||
98
docs/notes/overview.rst
Normal file
@@ -0,0 +1,98 @@
|
||||
:orphan:
|
||||
|
||||
.. _overview:
|
||||
|
||||
API Overview
|
||||
============
|
||||
|
||||
Below is a summary of Kaolin functionality. Refer to :ref:`tutorial_index` for specific use cases, examples
|
||||
and recipes that use these building blocks.
|
||||
|
||||
Operators for 3D Data:
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
:ref:`kaolin/ops<kaolin.ops>` contains operators for efficient processing functions of batched 3d models and tensors. We provide, conversions between 3d representations, primitives batching of heterogenenous data, and efficient mainstream functions on meshes and voxelgrids.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../modules/kaolin.ops
|
||||
|
||||
I/O:
|
||||
^^^^
|
||||
|
||||
:ref:`kaolin/io<kaolin.io>` contains functionality to interact with files.
|
||||
|
||||
We provide, importers and exporters to popular format such as .obj and .usd, but also utility functions and classes to preprocess and cache datasets with specific transforms.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../modules/kaolin.io
|
||||
|
||||
Metrics:
|
||||
^^^^^^^^
|
||||
|
||||
:ref:`kaolin/metrics<kaolin.metrics>` contains functions to compute distance and losses such as point_to_mesh distance, chamfer distance, IoU, or laplacian smoothing.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../modules/kaolin.metrics
|
||||
|
||||
Differentiable Rendering:
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
:ref:`kaolin/render<kaolin.render>` provide functions related to differentiable rendering, such a DIB-R rasterization, application of camera projection / translation / rotation, lighting, and textures.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../modules/kaolin.render
|
||||
|
||||
3D Checkpoints and Visualization:
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
:ref:`kaolin/visualize<kaolin.visualize>` contains utilities for writing 3D checkpoints for visualization. Currently we provide timelapse exporter that can be quickly picked up by the `Omniverse Kaolin App <https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/user_manual.html#training-visualizer>`_.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../modules/kaolin.visualize
|
||||
|
||||
Utilities:
|
||||
^^^^^^^^^^
|
||||
|
||||
:ref:`kaolin/utils<kaolin.utils>` contains utility functions to help development of application or research scripts. We provide functions to display and check informations about tensors, and features to fix seed.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../modules/kaolin.utils
|
||||
|
||||
Non Commercial
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
:ref:`kaolin/non_commercial<kaolin.non_commercial>` contains features under `NSCL license <https://github.com/NVIDIAGameWorks/kaolin/blob/master/LICENSE.NSCL>`_ restricted to non commercial usage for research and evaluation purposes.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../modules/kaolin.non_commercial
|
||||
|
||||
Licenses
|
||||
========
|
||||
|
||||
Most of Kaolin's repository is under `Apache v2.0 license <https://github.com/NVIDIAGameWorks/kaolin/blob/master/LICENSE>`_, except under :ref:`kaolin/non_commercial<kaolin.non_commercial>` which is under `NSCL license <https://github.com/NVIDIAGameWorks/kaolin/blob/master/LICENSE.NSCL>`_ restricted to non commercial usage for research and evaluation purposes. For example, FlexiCubes method is included under :ref:`non_commercial<kaolin.non_commercial>`.
|
||||
|
||||
Default `kaolin` import includes Apache-licensed components:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import kaolin
|
||||
|
||||
The non-commercial components need to be explicitly imported as:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import kaolin.non_commercial
|
||||
279
docs/notes/spc_summary.rst
Normal file
@@ -0,0 +1,279 @@
|
||||
Structured Point Clouds (SPCs)
|
||||
******************************
|
||||
|
||||
.. _spc:
|
||||
|
||||
Structured Point Clouds (SPC) is a sparse octree-based representation that is useful to organize and
|
||||
compress 3D geometrically sparse information.
|
||||
They are also known as sparse voxelgrids, quantized point clouds, and voxelized point clouds.
|
||||
|
||||
.. image:: ../img/mesh_to_spc.png
|
||||
|
||||
Kaolin supports a number of operations to work with SPCs,
|
||||
including efficient ray-tracing and convolutions.
|
||||
|
||||
The SPC data structure is very general. In the SPC data structure, octrees provide a way to store and efficiently retrieve coordinates of points at different levels of the octree hierarchy. It is also possible to associate features to these coordinates using point ordering in memory. Below we detail the low-level representations that comprise SPCs and allow corresponding efficient operations. We also provide a :ref:`convenience container<kaolin.rep>` for these low-level attributes.
|
||||
|
||||
Some of the conventions are also defined in `Neural Geometric Level of Detail: Real-time Rendering with
|
||||
Implicit 3D Surfaces <https://nv-tlabs.github.io/nglod/>`_ which uses SPC as an internal representation.
|
||||
|
||||
.. warning::
|
||||
Structured Point Clouds internal layout and structure is still experimental and may be modified in the future.
|
||||
|
||||
Octree
|
||||
======
|
||||
|
||||
.. _spc_octree:
|
||||
|
||||
Core to SPC is the `octree <https://en.wikipedia.org/wiki/Octree>`_, a tree data
|
||||
structure where each node have up to 8 childrens.
|
||||
We use this structure to do a recursive three-dimensional space partitioning,
|
||||
i.e: each node represents a partitioning of its 3D space (partition) of :math:`(2, 2, 2)`.
|
||||
The octree then contains the information necessary to find the sparse coordinates.
|
||||
|
||||
In SPC, a batch of octrees is represented as a tensor of bytes. Each bit in the byte array ``octrees`` represents
|
||||
the binary occupancy of an octree bit sorted in `Morton Order <https://en.wikipedia.org/wiki/Z-order_curve>`_.
|
||||
The Morton order is a type of space-filling curve which gives a deterministic ordering of
|
||||
integer coordinates on a 3D grid. That is, for a given non-negative 1D integer coordinate, there exists a
|
||||
bijective mapping to 3D integer coordinates.
|
||||
|
||||
Since a byte is a collection of 8 bits, a single byte ``octrees[i]``
|
||||
represents an octree node where each bit indicate the binary occupancy of a child node / partition as
|
||||
depicted below:
|
||||
|
||||
.. image:: ../img/octants.png
|
||||
:width: 600
|
||||
|
||||
For each octree, the nodes / bytes are following breadth-first-search order (with Morton order for
|
||||
childrens order), and the octree bytes are then :ref:`packed` to form ``octrees``. This ordering
|
||||
allows efficient tree access without having to explicilty store indirection pointers.
|
||||
|
||||
.. figure:: ../img/octree.png
|
||||
:scale: 30 %
|
||||
:alt: An octree 3D partitioning
|
||||
|
||||
Credit: https://en.wikipedia.org/wiki/Octree
|
||||
|
||||
The binary occupancy values in the bits of ``octrees`` implicitly encode position data due to the bijective
|
||||
mapping from Morton codes to 3D integer coordinates. However, to provide users a more straight
|
||||
forward interface to work with these octrees, SPC provides auxilary information such as
|
||||
``points`` which is a :ref:`packed` tensor of 3D coordinates. Refer to the :ref:`spc_attributes` section
|
||||
for more details.
|
||||
|
||||
Currently SPCs are primarily used to represent 3D surfaces,
|
||||
and so all the leaves are at the same ``level`` (depth).
|
||||
This allow very efficient processing on GPU, with custom CUDA kernels, for ray-tracing and convolution.
|
||||
|
||||
The structure contains finer details as you go deeper in to the tree.
|
||||
Below are the Levels 0 through 8 of a SPC teapot model:
|
||||
|
||||
.. image:: ../img/spcTeapot.png
|
||||
|
||||
Additional Feature Data
|
||||
=======================
|
||||
|
||||
The nodes of the ``octrees`` can contain information beyond just the 3D coordinates of the nodes,
|
||||
such as RGB color, normals, feature maps, or even differentiable activation maps processed by a
|
||||
convolution.
|
||||
|
||||
We follow a `Structure of Arrays <https://en.wikipedia.org/wiki/AoS_and_SoA>`_ approach to store
|
||||
additional data for maximum user extensibility.
|
||||
Currently the features would be tensors of shape :math:`(\text{num_nodes}, \text{feature_dim})`
|
||||
with ``num_nodes`` being the number of nodes at a specific ``level`` of the ``octrees``,
|
||||
and ``feature_dim`` the dimension of the feature set (for instance 3 for RGB color).
|
||||
Users can freely define their own feature data to be stored alongside SPC.
|
||||
|
||||
Conversions
|
||||
===========
|
||||
|
||||
Structured point clouds can be derived from multiple sources.
|
||||
|
||||
We can construct ``octrees``
|
||||
from unstructured point cloud data, from sparse voxelgrids
|
||||
or from the level set of an implicit function :math:`f(x, y, z)`.
|
||||
|
||||
.. _spc_attributes:
|
||||
|
||||
Related attributes
|
||||
==================
|
||||
|
||||
.. note::
|
||||
If you just wanna use the structured point clouds without having to go through the low level details, take a look at :ref:`the high level classes <kaolin.rep>`.
|
||||
|
||||
.. _spc_lengths:
|
||||
|
||||
``lengths:``
|
||||
------------
|
||||
|
||||
Since ``octrees`` use :ref:`packed` batching, we need ``lengths`` a 1D tensor of size ``batch_size`` that contains the size of each individual octree. Note that ``lengths.sum()`` should equal the size of ``octrees``. You can use :func:`kaolin.ops.batch.list_to_packed` to pack octrees and generate ``lengths``
|
||||
|
||||
.. _spc_pyramids:
|
||||
|
||||
``pyramids:``
|
||||
-------------
|
||||
|
||||
:class:`torch.IntTensor` of shape :math:`(\text{batch_size}, 2, \text{max_level} + 2)`. Contains layout information for each octree ``pyramids[:, 0]`` represent the number of points in each level of the ``octrees``, ``pyramids[:, 1]`` represent the starting index of each level of the octree.
|
||||
|
||||
.. _spc_exsum:
|
||||
|
||||
``exsum:``
|
||||
----------
|
||||
|
||||
:class:`torch.IntTensor` of shape :math:`(\text{octrees_num_bytes} + \text{batch_size})` is the exclusive sum of the bit counts of each ``octrees`` byte.
|
||||
|
||||
.. note::
|
||||
To generate ``pyramids`` and ``exsum`` see :func:`kaolin.ops.spc.scan_octrees`
|
||||
|
||||
.. _spc_points:
|
||||
|
||||
``point_hierarchies:``
|
||||
----------------------
|
||||
|
||||
:class:`torch.ShortTensor` of shape :math:`(\text{num_nodes}, 3)` correspond to the sparse coordinates at all levels. We refer to this :ref:`packed` tensor as the **structured point hierarchies**.
|
||||
|
||||
The image below show an analogous 2D example.
|
||||
|
||||
.. image:: ../img/spc_points.png
|
||||
:width: 400
|
||||
|
||||
the corresponding ``point_hierarchies`` would be:
|
||||
|
||||
>>> torch.ShortTensor([[0, 0], [1, 1],
|
||||
[1, 0], [2, 2],
|
||||
[2, 1], [3, 1], [5, 5]
|
||||
])
|
||||
|
||||
|
||||
.. note::
|
||||
To generate ``points`` see :func:`kaolin.ops.generate_points`
|
||||
|
||||
.. note::
|
||||
the tensors ``pyramid``, ``exsum`` and ``points`` are used by many Structured Point Cloud functions; avoiding their recomputation will improve performace.
|
||||
|
||||
|
||||
Convolutions
|
||||
============
|
||||
|
||||
We provide several sparse convolution layers for structured point clouds.
|
||||
Convolutions are characterized by the size of the input and output channels,
|
||||
an array of ``kernel_vectors``, and possibly the number of levels to ``jump``, i.e.,
|
||||
the difference in input and output levels.
|
||||
|
||||
.. _kernel-text:
|
||||
|
||||
An example of how to create a :math:`3 \times 3 \times 3` kernel follows:
|
||||
|
||||
>>> vectors = []
|
||||
>>> for i in range(-1, 2):
|
||||
>>> for j in range(-1, 2):
|
||||
>>> for k in range(-1, 2):
|
||||
>>> vectors.append([i, j, k])
|
||||
>>> Kvec = torch.tensor(vectors, dtype=torch.short, device=device)
|
||||
>>> Kvec
|
||||
tensor([[-1, -1, -1],
|
||||
[-1, -1, 0],
|
||||
[-1, -1, 1],
|
||||
...
|
||||
...
|
||||
[ 1, 1, -1],
|
||||
[ 1, 1, 0],
|
||||
[ 1, 1, 1]], device='cuda:0', dtype=torch.int16)
|
||||
|
||||
.. _neighborhood-text:
|
||||
|
||||
The kernel vectors determine the shape of the convolution kernel.
|
||||
Each kernel vector is added to the position of a point to determine
|
||||
the coordinates of points whose corresponding input data is needed for the operation.
|
||||
We formalize this notion using the following neighbor function:
|
||||
|
||||
.. math::
|
||||
|
||||
n(i,k) = \text{ID}\left(P_i+\overrightarrow{K}_k\right)
|
||||
|
||||
that returns the index of the point within the same level found by adding
|
||||
kernel vector :math:`\overrightarrow{K}_k` to point :math:`P_i`.
|
||||
Given the sparse nature of SPC data, it may be the case that no such point exists. In such cases, :math:`n(i,k)`
|
||||
will return an invalid value, and data accesses will be treated like zero padding.
|
||||
|
||||
Transposed convolutions are defined by the transposed neighbor function
|
||||
|
||||
.. math::
|
||||
|
||||
n^T(i,k) = \text{ID}\left(P_i-\overrightarrow{K}_k\right)
|
||||
|
||||
|
||||
The value **jump** is used to indicate the difference in levels between the iput features
|
||||
and the output features. For convolutions, this is the number of levels to downsample; while
|
||||
for transposed convolutions, **jump** is the number of levels to upsample. The value of **jump** must
|
||||
be positive, and may not go beyond the highest level of the octree.
|
||||
|
||||
Examples
|
||||
--------
|
||||
|
||||
You can create octrees from sparse feature_grids
|
||||
(of shape :math:`(\text{batch_size}, \text{feature_dim}, \text{height}, \text{width}, \text{depth})`):
|
||||
|
||||
>>> octrees, lengths, features = kaolin.ops.spc.feature_grids_to_spc(features_grids)
|
||||
|
||||
or from point cloud (of shape :math:`(\text{num_points, 3})`):
|
||||
|
||||
>>> qpc = kaolin.ops.spc.quantize_points(pc, level)
|
||||
>>> octree = kaolin.ops.spc.unbatched_points_to_octree(qpc, level)
|
||||
|
||||
To use convolution, you can use the functional or the torch.nn.Module version like torch.nn.functional.conv3d and torch.nn.Conv3d:
|
||||
|
||||
>>> max_level, pyramids, exsum = kaolin.ops.spc.scan_octrees(octrees, lengths)
|
||||
>>> point_hierarchies = kaolin.ops.spc.generate_points(octrees, pyramids, exsum)
|
||||
>>> kernel_vectors = torch.tensor([[0, 0, 0], [0, 0, 1], [0, 1, 0], [0, 1, 1],
|
||||
[1, 0, 0], [1, 0, 1], [1, 1, 0], [1, 1, 1]],
|
||||
dtype=torch.ShortTensor, device='cuda')
|
||||
>>> conv = kaolin.ops.spc.Conv3d(in_channels, out_channels, kernel_vectors, jump=1, bias=True).cuda()
|
||||
>>> # With functional
|
||||
>>> out_features, out_level = kaolin.ops.spc.conv3d(octrees, point_hierarchies, level, pyramids,
|
||||
... exsum, coalescent_features, weight,
|
||||
... kernel_vectors, jump, bias)
|
||||
>>> # With nn.Module and container class
|
||||
>>> input_spc = kaolin.rep.Spc(octrees, lengths)
|
||||
>>> conv
|
||||
>>> out_features, out_level = kaolin.ops.spc.conv_transpose3d(
|
||||
... **input_spc.to_dict(), input=out_features, level=level,
|
||||
... weight=weight, kernel_vectors=kernel_vectors, jump=jump, bias=bias)
|
||||
|
||||
To apply ray tracing we currently only support non-batched version, for instance here with RGB values as per point features:
|
||||
|
||||
>>> max_level, pyramids, exsum = kaolin.ops.spc.scan_octrees(
|
||||
... octree, torch.tensor([len(octree)], dtype=torch.int32, device='cuda')
|
||||
>>> point_hierarchy = kaolin.ops.spc.generate_points(octrees, pyramids, exsum)
|
||||
>>> ridx, pidx, depth = kaolin.render.spc.unbatched_raytrace(octree, point_hierarchy, pyramids[0], exsum,
|
||||
... origin, direction, max_level)
|
||||
>>> first_hits_mask = kaolin.render.spc.mark_pack_boundaries(ridx)
|
||||
>>> first_hits_point = pidx[first_hits_mask]
|
||||
>>> first_hits_rgb = rgb[first_hits_point - pyramids[max_level - 2]]
|
||||
|
||||
Going further with SPC:
|
||||
=======================
|
||||
|
||||
Examples:
|
||||
----------------------------
|
||||
|
||||
See our Jupyter notebook for a walk-through of SPC features:
|
||||
|
||||
`examples/tutorial/understanding_spcs_tutorial.ipynb <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/understanding_spcs_tutorial.ipynb>`_
|
||||
|
||||
And also our recipes for simple examples of how to use SPC:
|
||||
|
||||
* `spc_basics.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/spc/spc_basics.py>`_: showing attributes of an SPC object
|
||||
* `spc_dual_octree.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/spc/spc_dual_octree.py>`_: computing and explaining the dual of an SPC octree
|
||||
* `spc_trilinear_interp.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/spc/spc_trilinear_interp.py>`_: computing trilinear interpolation of a point cloud on an SPC
|
||||
|
||||
|
||||
SPC Documentation:
|
||||
------------------
|
||||
|
||||
Functions useful for working with SPCs are available in the following modules:
|
||||
|
||||
* :ref:`kaolin.ops.spc<kaolin.ops.spc>` - general explanation and operations
|
||||
* :ref:`kaolin.render.spc<kaolin.render.spc>` - rendering utilities
|
||||
* :class:`kaolin.rep.Spc` - high-level wrapper
|
||||
|
||||
|
||||
101
docs/notes/tutorial_index.rst
Normal file
@@ -0,0 +1,101 @@
|
||||
.. _tutorial_index:
|
||||
|
||||
Tutorial Index
|
||||
==============
|
||||
|
||||
Kaolin provides tutorials as ipython notebooks, docs pages and simple scripts. Note that the links
|
||||
point to master.
|
||||
|
||||
|
||||
Detailed Tutorials
|
||||
------------------
|
||||
|
||||
* `Camera and Rasterization <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/camera_and_rasterization.ipynb>`_: Rasterize ShapeNet mesh with nvdiffrast and camera:
|
||||
* Load ShapeNet mesh
|
||||
* Preprocess mesh and materials
|
||||
* Create a camera with ``from_args()`` general constructor
|
||||
* Render a mesh with multiple materials with nvdiffrast
|
||||
* Move camera and see the resulting rendering
|
||||
* `Optimizing Diffuse Lighting <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/diffuse_lighting.ipynb>`_: Optimize lighting parameters with spherical gaussians and spherical harmonics:
|
||||
* Load an obj mesh with normals and materials
|
||||
* Rasterize the diffuse and specular albedo
|
||||
* Render and optimize diffuse lighting:
|
||||
* Spherical harmonics
|
||||
* Spherical gaussian with inner product implementation
|
||||
* Spherical gaussian with fitted approximation
|
||||
* `Optimize Diffuse and Specular Lighting with Spherical Gaussians <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/sg_specular_lighting.ipynb>`_:
|
||||
* Load an obj mesh with normals and materials
|
||||
* Generate view rays from camera
|
||||
* Rasterize the diffuse and specular albedo
|
||||
* Render and optimize diffuse and specular lighting with spherical gaussians
|
||||
* `Working with Surface Meshes <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/working_with_meshes.ipynb>`_:
|
||||
* loading and constructing :class:`kaolin.rep.SurfaceMesh` objects
|
||||
* batching of meshes
|
||||
* auto-computing common attributes (like ``face_normals``)
|
||||
* `Deep Marching Tetrahedra <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/dmtet_tutorial.ipynb>`_: reconstructs a tetrahedral mesh from point clouds with `DMTet <https://nv-tlabs.github.io/DMTet/>`_, covering:
|
||||
* generating data with Omniverse Kaolin App
|
||||
* loading point clouds from a ``.usd`` file
|
||||
* chamfer distance as a loss function
|
||||
* differentiable marching tetrahedra
|
||||
* using Timelapse API for 3D checkpoints
|
||||
* visualizing 3D results of training
|
||||
* `Understanding Structured Point Clouds (SPCs) <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/understanding_spcs_tutorial.ipynb>`_: walks through SPC features, covering:
|
||||
* under-the-hood explanation of SPC, why it's useful and key ops
|
||||
* loading a mesh
|
||||
* sampling a point cloud
|
||||
* converting a point cloud to SPC
|
||||
* setting up camera
|
||||
* rendering SPC with ray tracing
|
||||
* storing features in an SPC
|
||||
* `Differentiable Rendering <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/dibr_tutorial.ipynb>`_: optimizes a triangular mesh from images using `DIB-R <https://github.com/nv-tlabs/DIB-R-Single-Image-3D-Reconstruction>`_ renderer, covering:
|
||||
* generating data with Omniverse Kaolin App, and loading this synthetic data
|
||||
* loading a mesh
|
||||
* computing mesh laplacian
|
||||
* DIB-R rasterization
|
||||
* differentiable texture mapping
|
||||
* computing mask intersection-over-union loss (IOU)
|
||||
* using Timelapse API for 3D checkpoints
|
||||
* visualizing 3D results of training
|
||||
* `Fitting a 3D Bounding Box <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/bbox_tutorial.ipynb>`_: fits a 3D bounding box around an object in images using `DIB-R <https://github.com/nv-tlabs/DIB-R-Single-Image-3D-Reconstruction>`_ renderer, covering:
|
||||
* generating data with Omniverse Kaolin App, and loading this synthetic data
|
||||
* loading a mesh
|
||||
* DIB-R rasterization
|
||||
* computing mask intersection-over-union loss (IOU)
|
||||
* :ref:`3d_viz`: explains saving 3D checkpoints and visualizing them, covering:
|
||||
* using Timelapse API for writing 3D checkpoints
|
||||
* understanding output file format
|
||||
* visualizing 3D checkpoints using Omniverse Kaolin App
|
||||
* visualizing 3D checkpoints using bundled ``kaolin-dash3d`` commandline utility
|
||||
* `Reconstructing Point Cloud with DMTet <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/dmtet_tutorial.ipynb>`_: Trains an SDF estimator to reconstruct a mesh from a point cloud covering:
|
||||
* using point clouds data generated with Omniverse Kaolin App
|
||||
* loading point clouds from an USD file.
|
||||
* defining losses and regularizer for a mesh with point cloud ground truth
|
||||
* applying marching tetrahedra
|
||||
* using Timelapse API for 3D checkpoints
|
||||
* visualizing 3D checkpoints using ``kaolin-dash3d``
|
||||
|
||||
|
||||
Simple Recipes
|
||||
--------------
|
||||
|
||||
* I/O and Data Processing:
|
||||
* `usd_kitchenset.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/usd_kitchenset.py>`_: loading multiple meshes from a ``.usd`` file and saving
|
||||
* `spc_from_pointcloud.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/dataload/spc_from_pointcloud.py>`_: converting a point cloud to SPC object
|
||||
* `occupancy_sampling.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/preprocess/occupancy_sampling.py>`_: computing occupancy function of points in a mesh using ``check_sign``
|
||||
* `spc_basics.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/spc/spc_basics.py>`_: showing attributes of an SPC object
|
||||
* `spc_dual_octree.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/spc/spc_dual_octree.py>`_: computing and explaining the dual of an SPC octree
|
||||
* `spc_trilinear_interp.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/spc/spc_trilinear_interp.py>`_: computing trilinear interpolation of a point cloud on an SPC
|
||||
* Visualization:
|
||||
* `visualize_main.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/visualize_main.py>`_: using Timelapse API to write mock 3D checkpoints
|
||||
* `fast_mesh_sampling.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/preprocess/fast_mesh_sampling.py>`_: Using CachedDataset to preprocess a ShapeNet dataset we can sample point clouds efficiently at runtime
|
||||
* Camera:
|
||||
* `cameras_differentiable.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/cameras_differentiable.py>`_: optimize a camera position
|
||||
* `camera_transforms.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_transforms.py>`_: using :func:`Camera.transform()` function
|
||||
* `camera_ray_tracing.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_ray_tracing.py>`_: how to design a ray generating function using :class:`Camera` objects
|
||||
* `camera_properties.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_properties.py>`_: exposing some the camera attributes and properties
|
||||
* `camera_opengl_shaders.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_opengl_shaders.py>`_: Using the camera with glumpy
|
||||
* `camera_movement.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_movement.py>`_: Manipulating a camera position and zoom
|
||||
* `camera_init_simple.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_init_simple.py>`_: Making Camera objects with the flexible :func:`Camera.from_args()` constructor
|
||||
* `camera_init_explicit.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_init_explicit.py>`_: Making :class:`CameraIntrinsics` and :class:`CameraExtrinsics` with all the different constructors available
|
||||
* `camera_coordinate_systems.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/recipes/camera/camera_coordinate_systems.py>`_: Changing coordinate system in a :class:`Camera` object
|
||||
|
||||
4
docs/readthedocs_requirements.txt
Normal file
@@ -0,0 +1,4 @@
|
||||
numpy<1.27.0,>=1.19.5
|
||||
scipy==1.10.1
|
||||
-f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
|
||||
torch==1.8.2+cpu
|
||||