diff --git a/docs/source/contributing/testing.rst b/docs/source/contributing/testing.rst index dfc6a225f8..8013fac254 100644 --- a/docs/source/contributing/testing.rst +++ b/docs/source/contributing/testing.rst @@ -292,10 +292,10 @@ Note the fixtures here: You have to generate a ``.json`` file first to be able to test your video. To do that, use ``helpers.save_control_data_from_video``. -For instance, a test that will check if the l flag works properly will first -require rendering a video using the -l flag from a scene. Then we will test -(in this case, SquareToCircle), that lives in -``test_scene_rendering/simple_scene.py``. Change directories to ``tests/``, +For instance, testing low-quality output requires first rendering a video from a +scene with the ``-ql`` flag. The example below tests +``SquareToCircle``, which lives in ``test_scene_rendering/simple_scene.py``. +Change directories to ``tests/``, create a file (e.g. ``create\_data.py``) that you will remove as soon as you're done. Then run: diff --git a/docs/source/faq/general.md b/docs/source/faq/general.md index 8ac092f5f1..f8286396fc 100644 --- a/docs/source/faq/general.md +++ b/docs/source/faq/general.md @@ -76,8 +76,9 @@ ones that are documented for the base classes {class}`.VMobject` and Yes: simply pass the CLI flag `-t` (or its long form `--transparent`). Note that the default video file format does not support transparency, which is why Manim will output a `.mov` instead of a `.mp4` when -rendering with a transparent background. Other movie file formats -that support transparency can be obtained by passing +rendering with a transparent background. Explicitly requesting +`--format=mp4` together with transparency is rejected. Other movie file +formats that support transparency can be obtained by passing `--format=webm` or `--format=gif`. --- diff --git a/docs/source/guides/configuration.rst b/docs/source/guides/configuration.rst index 3ea27b89db..4b5f8279d4 100644 --- a/docs/source/guides/configuration.rst +++ b/docs/source/guides/configuration.rst @@ -37,11 +37,20 @@ An example of using the above form is: This asks Manim to search for a Scene class called :code:`SceneOne` inside the file ``file.py`` and render it with medium quality (specified by the ``-qm`` flag). -Another frequently used flag is ``-p`` ("preview"), which makes manim -open the rendered video after it's done rendering. +Another frequently used flag is ``-p`` ("preview"), which makes Manim +open the rendered artifact after rendering. This behavior is the same for all +renderers. -.. note:: The ``-p`` flag does not change any properties of the global - ``config`` dict. The ``-p`` flag is only a command-line convenience. +Renderers may also support a live preview while the scene is evaluated. The +OpenGL renderer provides this with ``-l`` (or ``--live-preview``): + +.. code-block:: bash + + manim --renderer=opengl -l SceneName + +With ``--format=auto``, live preview does not write a media file, keeping the +interactive workflow fast. Pass a concrete format, such as ``--format=mp4``, +to record the scene while displaying the live preview. Advanced examples ================= @@ -53,6 +62,10 @@ instead of the whole video, you can execute manim -sqh SceneName +The equivalent ``--format=png`` spelling uses the same fast mode that saves only +the last frame. Use ``--format=png-sequence`` when every rendered frame should +be written as a numbered PNG instead. + The following example specifies the output file name (with the :code:`-o` flag), renders only the first ten animations (:code:`-n` flag) with a white background (:code:`-c` flag), and saves the animation as a ``.gif`` instead of as a @@ -161,17 +174,36 @@ and serve the same purpose. Take, for example, the following config file. [CLI] # my config file output_file = myscene - save_as_gif = True + format = gif background_color = WHITE Config files are parsed with the standard python library ``configparser``. In particular, they will ignore any line that starts with a pound symbol ``#``. +Video encoder profiles use dedicated sections because codec options are arbitrary +key/value pairs: + +.. code-block:: ini + + [video_encoder] + codec = libx264 + pixel_format = yuv420p + + [video_encoder.options] + crf = 18 + preset = slow + +``codec`` and ``pixel_format`` default to ``auto``. Manim resolves them from the +output format and whether alpha is required. The equivalent CLI options are +``--video-codec``, ``--pixel-format``, and repeatable +``--encoder-option KEY=VALUE``. If any ``--encoder-option`` is supplied, the CLI +option map replaces the complete ``[video_encoder.options]`` map. + Now, executing the following command .. code-block:: bash - manim -o myscene -i -c WHITE SceneName + manim -o myscene --format=gif -c WHITE SceneName is equivalent to executing the following command, provided that ``manim.cfg`` is in the same directory as , @@ -253,7 +285,7 @@ For example, take the following user-wide config file # user-wide [CLI] output_file = myscene - save_as_gif = True + format = gif background_color = WHITE and the following folder-wide file @@ -262,7 +294,7 @@ and the following folder-wide file # folder-wide [CLI] - save_as_gif = False + format = auto Then, executing :code:`manim SceneName` will be equivalent to not using any config files and executing @@ -344,26 +376,17 @@ highest precedence is: 5. any programmatic changes made after the config system is set. -A list of all config options -**************************** +Inspecting available config options +*********************************** -.. code:: +Run ``manim cfg show`` to inspect the currently resolved configuration and +``manim render --help`` for the authoritative list of render CLI options. The +attributes available for programmatic configuration are documented on +:class:`.ManimConfig`. - ['aspect_ratio', 'assets_dir', 'background_color', 'background_opacity', - 'bottom', 'custom_folders', 'disable_caching', 'dry_run', - 'encoder_queue_size', 'ffmpeg_loglevel', 'flush_cache', 'frame_height', 'frame_rate', - 'frame_size', 'frame_width', 'frame_x_radius', 'frame_y_radius', - 'from_animation_number', `fullscreen`, 'images_dir', 'input_file', 'left_side', - 'log_dir', 'log_to_file', 'max_files_cached', 'max_inflight_encoders', - 'media_dir', 'media_width', 'movie_file_extension', 'notify_outdated_version', - 'output_file', 'partial_movie_dir', - 'pixel_height', 'pixel_width', 'plugins', 'preview', - 'progress_bar', 'quality', 'right_side', 'save_as_gif', 'save_last_frame', - 'save_pngs', 'scene_names', 'show_in_file_browser', 'sound', 'tex_dir', - 'tex_template', 'tex_template_file', 'text_dir', 'top', 'transparent', - 'upto_animation_number', 'use_opengl_renderer', 'verbosity', 'video_dir', - 'window_position', 'window_monitor', 'window_size', 'write_all', 'write_to_movie', - 'enable_wireframe', 'force_window'] +Some CLI conveniences intentionally map to a canonical configuration value. In +particular, ``-s`` / ``--save_last_frame`` sets ``format = png``; configuration +files should use the canonical ``format`` option directly. Accessing CLI command options diff --git a/docs/source/guides/deep_dive.rst b/docs/source/guides/deep_dive.rst index 050eb052d5..903acee8e2 100644 --- a/docs/source/guides/deep_dive.rst +++ b/docs/source/guides/deep_dive.rst @@ -119,10 +119,9 @@ Preliminaries Importing the library ^^^^^^^^^^^^^^^^^^^^^ -Independent of how exactly you are telling your system -to render the scene, i.e., whether you run ``manim -qm -p file_name.py ToyExample``, or -whether you are rendering the scene directly from the Python script via a snippet -like +You can ask Manim to render in several ways. For example, you can run +``manim -qm -p file_name.py ToyExample`` or render directly from a Python script +with a snippet like :: @@ -130,8 +129,9 @@ like scene = ToyExample() scene.render() -or whether you are rendering the code in a Jupyter notebook, you are still telling your -python interpreter to import the library. The usual pattern used to do this is +Whether you are rendering this way or from a Jupyter notebook, you are still +telling your Python interpreter to import the library. The usual pattern used to +do this is :: @@ -243,7 +243,7 @@ then calls the scene's render method in the **Jupyter notebooks.** In Jupyter notebooks, the communication with the library is handled by the ``%%manim`` magic command, which is implemented in the ``manim.utils.ipython_magic`` module. There is -:meth:`some documentation <.ManimMagic.manim>` available for the magic command, +:meth:`some documentation <.ManimMagic.manim>` available for the magic command. The implementation instantiates the requested scene and calls its render method; the scene attaches a manager lazily through that entry point. @@ -263,18 +263,72 @@ depend on any configuration options set in ``config``. Then the scene inspects t ``config.renderer``, and based on its value, either instantiates a ``CairoRenderer`` or an ``OpenGLRenderer`` object and assigns it to its ``renderer`` attribute. -The scene then asks its renderer to initialize the scene by calling +After selecting the renderer, the scene resolves the mutable configuration into +one immutable render-session specification. In abbreviated form, initialization +continues as follows: :: - self.renderer.init_scene(self) - -Inspecting both the default Cairo renderer and the OpenGL renderer shows that the ``init_scene`` -method effectively makes the renderer instantiate a :class:`.SceneFileWriter` object, which -basically is Manim's interface to ``libav`` (FFMPEG) and actually writes the movie file. The Cairo -renderer (see the implementation `here `__) does not require any further initialization. The OpenGL renderer -does some additional setup to enable the realtime rendering preview window, which we do not go -into detail further here. + self.session_spec = resolve_render_session( + config, + self.renderer.capabilities, + renderer_name=type(self.renderer).__name__, + ) + self.output_plan = resolve_output_plan( + resolve_media_layout(...), + self.session_spec.output, + scene_name=type(self).__name__, + requested_output_name=..., + ) + self.file_writer_settings = _SceneFileWriterSettings( + plan=self.output_plan, + video_encoder=self.session_spec.video_encoder, + max_inflight_encoders=config.max_inflight_encoders, + encoder_queue_size=config.encoder_queue_size, + max_files_cached=config.max_files_cached, + assets_dir=..., + ) + self.renderer.init_scene( + self, + self.session_spec, + self.file_writer_settings, + ) + +The session specification separates primary artifact intent (an ``OutputSpec``) +from presentation requests such as opening the completed artifact or displaying +a live preview. For video output it also contains the resolved segment profile: +container, codec, pixel format, dimensions, exact frame rate, and codec options. +It records dry-run execution separately from artifact selection. A dry run +requests semantic scene evaluation without rasterizing +frames or using media and cache resources. In contrast, ``format = none`` only +suppresses the primary artifact; an OpenGL live preview with automatic output +still rasterizes and displays frames without writing a file. Both requests have +an effective output format of ``none``, so the session's ``dry_run`` field +preserves the intended execution behavior. + +The current renderer loops do not yet enforce no-raster dry runs; the session +field preserves that request for Manager-owned execution. Resolution also +validates requests against the selected renderer's capabilities. For example, +Cairo rejects live preview, while OpenGL advertises support for it. A concrete +format records the live preview as well. + +The scene resolves configured directory templates once into an immutable output +plan containing exact scene-specific artifact, section, image-sequence, and cache +paths. Planning performs no file I/O and creates no directories. ``output_file`` +supplies the artifact name, while the resolved format supplies its suffix. + +The scene combines the output plan and segment profile with the encoder-pool, +cache-maintenance, and sound-asset inputs in immutable +``_SceneFileWriterSettings``. Both renderers instantiate a +:class:`.SceneFileWriter` from these settings. The writer does not retain a +renderer reference or read mutable global configuration. Directories are created +lazily when their owning operation first writes. The Cairo renderer (see the +implementation `here +`__) +does not require further renderer-specific initialization. OpenGL creates a +window only when the resolved presentation specification requests a live preview. +The ``-p`` / ``--preview`` option does not create this window; it opens the +completed artifact after rendering. After the renderer has been instantiated and initialized its file writer, the scene populates further initial attributes (notable mention: the ``mobjects`` attribute @@ -283,10 +337,10 @@ attribute is initially ``None`` unless the caller attaches a manager explicitly. .. warning:: - :class:`.Manager` is an incremental coordination boundary. At this stage the - renderer still owns its camera, clock, play count, skip state, and file writer; - the manager exposes forwarding views of them. The scene and renderer therefore - still have substantial interplay that later refactors aim to remove. + The scene captures the immutable session specification and output plan before + renderer initialization. The manager coordinates the scene lifecycle, while + the renderer still owns its camera, clock, play count, skip state, and file + writer. The manager currently exposes these through forwarding properties. The rest of this article is concerned with the last line in our toy example script:: @@ -312,13 +366,19 @@ The first three call the corresponding customizable scene hooks: hook is more relevant for situations where Manim is used within other Python scripts). -After these hooks have run, :meth:`.Manager.post_construct` asks the renderer to -finish the scene. For Cairo this calls :meth:`.CairoRenderer.scene_finished`, -which checks whether animations have been played and tells the -:class:`.SceneFileWriter` to finish the output. For video output, the file writer -waits for partial movie files that are still being encoded and combines them into -the final movie. If no animations have been played, Manim assumes that a static -image should be output. +After these hooks have run, :meth:`.Manager.post_construct` checks whether the +resolved output can be finalized. An explicit video format for a scene without +play calls raises an error instead of silently changing the requested artifact. +Otherwise, the manager asks the renderer to finish the scene. For Cairo this +calls :meth:`.CairoRenderer.scene_finished`, which tells the +:class:`.SceneFileWriter` to finish any time-based output. For video output, the +file writer waits for partial movie files that are still being encoded and +combines them into the final movie. Last-frame PNG output fast-forwards +animations and renders a single image. When automatic output selected a video +for a scene without play calls, the writer saves that image as a fallback PNG and +the manager logs a warning. The writer records the completed artifact as +``final_file_path``. After finalization, the manager uses the presentation +specification to open the artifact or reveal it in the file browser. **Back in our toy example,** the call to :meth:`.Scene.render` creates a manager, then :meth:`.Manager.render` triggers :meth:`.Scene.setup` (which only consists of @@ -767,13 +827,15 @@ to learn more, the :func:`.get_hash_from_play_call` function in the :mod:`.utils.hashing` module is essentially the entry point to the caching mechanism. -In the event that the animation has to be rendered, the renderer asks -its :class:`.SceneFileWriter` to open a partial movie stream. The file writer -uses ``libav`` to create a container and video stream, then wraps them in a -``_PartialMovieEncodeJob``. Each encoding job owns its container, stream, frame -queue, and worker thread. During the render loop, rendered raw frames are added -to this queue and encoded by the worker. With the writing process in place, the -renderer then asks the scene to "begin" the animations. +In the event that the animation has to be rendered, the renderer gives its +:class:`.SceneFileWriter` the current animation index and asks it to start a +segment job. The writer creates a ``VideoSegmentEncoder`` from the resolved +profile and wraps it in a ``_PartialMovieEncodeJob``. The synchronous segment +encoder owns its container, video stream, sequential presentation timestamps, +and target cleanup. The job owns only the frame queue and worker thread. During +the render loop, concrete top-left-origin ``uint8`` RGBA arrays are added to the +queue and encoded by the worker. With the writing process in place, the renderer +then asks the scene to "begin" the animations. By default, Manim finishes encoding each partial movie file before rendering the next animation. If ``max_inflight_encoders`` is set to a value greater than 1, @@ -826,9 +888,8 @@ time is extracted (3 seconds long) and stored in ``Scene.duration``. The renderer then checks whether it should skip (it should not), then whether the animation is already cached (it is not). The corresponding animation hash value is -determined and passed to the file writer, which then also calls -``libav`` to start the writing process which waits for rendered -frames from the library. +determined and passed to the file writer. The writer resolves the segment target +from that key and starts its queued encoder, which waits for rendered frames. The scene then ``begin``\ s the animation: for the :class:`.ReplacementTransform` this means that the animation populates @@ -968,8 +1029,10 @@ camera is asked to capture the scene: After all batches have been processed, the camera has an image representation of the Scene at the current time stamp in form of a NumPy array stored in its -``pixel_array`` attribute. The renderer then takes this array and passes it to -its :class:`.SceneFileWriter`. This concludes one iteration of the render loop, +``pixel_array`` attribute. The renderer passes a top-left-origin, +C-contiguous ``uint8`` RGBA array to its :class:`.SceneFileWriter`. OpenGL uses +the same array contract and performs GPU readback at this renderer boundary only +when file output needs a frame. This concludes one iteration of the render loop, and once the time progression has been processed completely, a final bit of cleanup is performed before the :meth:`.Scene.play_internal` call is completed. @@ -1038,9 +1101,11 @@ calls the scene's cleanup method :meth:`.Scene.tear_down`, followed by scene, and the renderer in turn asks its scene file writer to wrap things up by calling :meth:`.SceneFileWriter.finish`. The file writer first waits for all remaining encoding jobs, then combines the completed partial movie files into the -final product. If rendering aborts during a play instead, the manager asks the file -writer to abort its encoding jobs; the incomplete current partial movie file is -removed so that it cannot be mistaken for a valid cached result on a later render. +final product and records its path. If rendering aborts during a play instead, the +manager asks the file writer to abort its encoding jobs; the incomplete current +partial movie file is removed so that it cannot be mistaken for a valid cached +result on a later render. Once output is finalized, the manager carries out any +post-render presentation request from the immutable session specification. And there you go! This is a more or less detailed description of how Manim works under the hood. While we did not discuss every single line of code in detail diff --git a/docs/source/installation/docker.rst b/docs/source/installation/docker.rst index c374b08537..67f05eb595 100644 --- a/docs/source/installation/docker.rst +++ b/docs/source/installation/docker.rst @@ -14,9 +14,9 @@ For our image ``manimcommunity/manim``, there are the following tags: .. note:: - When using Manim's CLI within a Docker container, some flags like - ``-p`` (preview file) and ``-f`` (show output file in the file browser) - are not supported. + When using Manim's CLI within a Docker container, options that launch host + applications, such as ``-p`` and ``--show_in_file_browser``, are not + supported. Live preview also requires explicit display forwarding. .. note:: diff --git a/docs/source/tutorials/output_and_config.rst b/docs/source/tutorials/output_and_config.rst index af7961d873..e2e92a2bd7 100644 --- a/docs/source/tutorials/output_and_config.rst +++ b/docs/source/tutorials/output_and_config.rst @@ -37,7 +37,8 @@ files and the project folder will look as follows. | └─480p15 | ├─SquareToCircle.mp4 | └─partial_movie_files - ├─text + | └─SquareToCircle + ├─texts └─Tex @@ -45,12 +46,13 @@ There are quite a few new files. The main output is in ``media/videos/scene/480p15/SquareToCircle.mp4``. By default, the ``media`` folder will contain all of manim's output files. The ``media/videos`` subfolder contains the rendered videos. Inside of it, you will find one folder -for each different video quality. In our case, since we used the ``-l`` flag, +for each different video quality. In our case, since we used the ``-ql`` flag, the video was generated at 480 resolution at 15 frames per second from the ``scene.py`` file. Therefore, the output can be found inside ``media/videos/scene/480p15``. The additional folders -``media/videos/scene/480p15/partial_movie_files`` as well as ``media/text`` and -``media/Tex`` contain files that are used by manim internally. +``media/videos/scene/480p15/partial_movie_files/SquareToCircle`` as well as +``media/texts`` and ``media/Tex`` contain files that are used by Manim +internally. You can see how manim makes use of the generated folder structure by executing the following command, @@ -93,16 +95,18 @@ And the folder structure should look as follows. | ├─480p15 | | ├─SquareToCircle.mp4 | | └─partial_movie_files + | | └─SquareToCircle | └─1080p60 | ├─SquareToCircle.mp4 | └─partial_movie_files - ├─text + | └─SquareToCircle + ├─texts └─Tex Manim has created a new folder ``media/videos/1080p60``, which corresponds to -the high resolution and the 60 frames per second. Inside of it, you can find +the high resolution and the 60 frames per second. Inside of it, you can find the new ``SquareToCircle.mp4``, as well as the corresponding -``partial_movie_files``. +``partial_movie_files/SquareToCircle`` directory. When working on a project with multiple scenes, and trying out multiple resolutions, the structure of the output directories will keep all your videos @@ -119,22 +123,94 @@ The corresponding folder structure looks like this: └─media ├─images | └─scene - | ├─SquareToCircle.png + | ├─SquareToCircle_ManimCE_vX.Y.Z.png ├─videos | └─scene | ├─480p15 | | ├─SquareToCircle.mp4 | | └─partial_movie_files + | | └─SquareToCircle | └─1080p60 | ├─SquareToCircle.mp4 | └─partial_movie_files - ├─text + | └─SquareToCircle + ├─texts └─Tex -Saving the last frame with ``-s`` can be combined with the flags for different -resolutions, e.g. ``-s -ql``, ``-s -qh`` - +``X.Y.Z`` in the PNG filename stands for the installed Manim version. Manim +adds this version suffix to GIF names and PNG files produced with ``-s`` when no +explicit ``--output_file`` is supplied. +Saving the last frame with ``-s`` can be combined with the flags for different +resolutions, e.g. ``-s -ql``, ``-s -qh``. The equivalent +``--format=png`` spelling uses the same fast mode that saves only the last frame. +To write every rendered frame as a numbered PNG instead, use +``--format=png-sequence``. The sequence is stored in a scene-specific directory, +for example ``media/images/scene/SquareToCircle/0000.png``. + +Customizing output directories +****************************** + +The canonical layout can be customized through the ordinary directory options in +the ``[CLI]`` section of ``manim.cfg``. These options support placeholders and may +refer to one another; for example: + +.. code-block:: ini + + [CLI] + media_dir = project-media + video_dir = {media_dir}/renders/{module_name}/{quality} + images_dir = {media_dir}/stills/{module_name} + sections_dir = {video_dir}/sections + partial_movie_dir = {video_dir}/partial_movie_files/{scene_name} + log_dir = {media_dir}/logs + +Output formats +************** + +The ``--format`` option selects one primary artifact: + +.. list-table:: + :header-rows: 1 + + * - Value + - Behavior + * - ``auto`` + - The default. Produces MP4 for opaque output and MOV for transparent + output. A scene without play or wait calls produces a last-frame PNG + instead and logs a warning. When live preview is requested, it produces + no file unless a concrete format is given. + * - ``mp4``, ``mov``, ``webm`` or ``gif`` + - Produces the selected video format. A scene without play or wait calls + cannot produce an explicitly requested video. Transparent MP4 is rejected + because that container does not support alpha transparency. + * - ``png`` + - Fast-forwards animations and writes only the last frame of the scene. + This is equivalent to ``-s``. + * - ``png-sequence`` + - Evaluates the full frame progression and writes every frame as PNG. + * - ``none`` + - Evaluates the scene without producing a primary media artifact. This + controls artifact persistence only: live preview, for example, still + rasterizes and presents frames while the resolved format is ``none``. + +``--dry_run`` also suppresses media output and cannot be combined with live +preview, but it counts as a separate execution request rather than modifying the +choice of output format. This allows Manim to function as if it were rendering a +normal scene, but without producing any artifact. + +Video output is assembled from silent cached segments. Manim selects their codec +and pixel format automatically; ``--video-codec``, ``--pixel-format``, and +repeatable ``--encoder-option KEY=VALUE`` provide explicit control when needed. +These settings are part of segment cache identity, so changing one rerenders the +affected segments. Audio is mixed into the final artifact separately. + +``-o`` / ``--output_file`` names the primary artifact for a single selected scene; +``--format`` selects its format. Manim appends the resolved format suffix unless +the name already ends with it. For example, ``-o movie.mp4 --format=mp4`` produces +``movie.mp4``, while ``-o movie.mov --format=mp4`` produces ``movie.mov.mp4``. A +single output name is ambiguous for a multi-scene render, so ``-o`` cannot be +combined with ``--write_all`` or with several selected scene names. Sections @@ -155,7 +231,10 @@ its own output video. The cuts between two sections can be set like this: self.next_section("this is a section without any animations, it will be removed") All the animations between two of these cuts get concatenated into a single output -video file. +video file. The original section name is retained in the section metadata. Its output +filename uses a safe slug, so a name such as ``"create square"`` appears as +``create-square`` in the filename. + Be aware that you need at least one animation in each section. For example this wouldn't create an output video: .. code-block:: python @@ -206,11 +285,15 @@ If you do this, the ``media`` folder will look like this: │ ├── 3163782288_524160878_1793580042.mp4 │ └── partial_movie_file_list.txt └── sections - ├── ElaborateSceneWithSections_0000.mp4 - ├── ElaborateSceneWithSections_0001.mp4 - ├── ElaborateSceneWithSections_0002.mp4 + ├── ElaborateSceneWithSections_0000_create-square.mp4 + ├── ElaborateSceneWithSections_0001_transform-to-circle.mp4 + ├── ElaborateSceneWithSections_0003_fade-out.mp4 └── ElaborateSceneWithSections.json +The ``partial_movie_file_list.txt`` file records the complete segment order used for +main movie assembly and can help diagnose concatenation problems. Section assembly +does not overwrite this diagnostic list. + As you can see each section receives their own output video in the ``sections`` directory. The JSON file in here contains some useful information for each section: @@ -289,14 +372,15 @@ prototyping and testing. The other options that specify render quality are (1920x1080 60FPS), 2k (2560x1440 60FPS) and 4k quality (3840x2160 60FPS), respectively. -The ``-p`` flag plays the animation once it is rendered. If you want to open -the file browser at the location of the animation instead of playing it, you -can use the ``-f`` flag. You can also omit these two flags. +The ``-p`` flag opens the animation once it is rendered. +``--show_in_file_browser`` reveals the artifact in the file browser. The options +can be combined; Manim reveals the artifact first and then opens it for preview. -Finally, by default manim will output .mp4 files. If you want your animations -in .gif format instead, use the ``--format gif`` flag. The output files will -be in the same folder as the .mp4 files, and with the same name, but a -different file extension. +The separate ``-l`` (or ``--live-preview``) option asks a capable renderer to +display frames while rendering. The OpenGL renderer supports this mode. Live +preview with the default ``--format=auto`` does not produce a media file; you +must pass a concrete format such as ``--format=mp4`` to display and record +simultaneously. This was a quick review of some of the most frequent command-line flags. For a thorough review of all flags available, see the :doc:`thematic guide on diff --git a/manim/__main__.py b/manim/__main__.py index 572053bf08..3b0247ab47 100644 --- a/manim/__main__.py +++ b/manim/__main__.py @@ -5,6 +5,7 @@ from manim import __version__ from manim._config import cli_ctx_settings, console +from manim.cli.cache.commands import cache from manim.cli.cfg.group import cfg from manim.cli.checkhealth.commands import checkhealth from manim.cli.default_group import DefaultGroup @@ -92,6 +93,7 @@ def main(ctx: click.Context) -> None: main.add_command(checkhealth) +main.add_command(cache) main.add_command(cfg) main.add_command(plugins) main.add_command(init) diff --git a/manim/_config/default.cfg b/manim/_config/default.cfg index a77e832279..c573c491a5 100644 --- a/manim/_config/default.cfg +++ b/manim/_config/default.cfg @@ -5,46 +5,32 @@ # specifying any flags [CLI] -# Each of the following will be set to True if the corresponding CLI flag -# is present when executing manim. If the flag is not present, they will -# be set to the value found here. For example, running manim with the -w -# flag will set WRITE_TO_MOVIE to True. However, since the default value -# of WRITE_TO_MOVIE defined in this file is also True, running manim -# without the -w value will also output a movie file. To change that, set -# WRITE_TO_MOVIE = False so that running manim without the -w flag will not -# generate a movie file. Note all of the following accept boolean values. +# CLI values override the settings found here. Optional CLI flags preserve +# configured values when they are not explicitly passed. # --notify_outdated_version notify_outdated_version = True -# -w, --write_to_movie -write_to_movie = True - -format = mp4 - -# -s, --save_last_frame -# setting save_last_frame to True forces write_to_movie to False -save_last_frame = False +# Primary output format. "auto" resolves to mp4 for opaque scenes and mov for +# transparent scenes. Use "none" to disable media output. +format = auto # -a, --write_all write_all = False -# -g, --save_pngs -save_pngs = False - # -0, --zero_pad zero_pad = 4 -# -i, --save_as_gif -save_as_gif = False - # --save_sections save_sections = False -# -p, --preview +# -p, --preview: open the completed artifact after rendering preview = False -# -f, --show_in_file_browser +# -l, --live-preview: display frames while rendering when supported +live_preview = False + +# --show_in_file_browser show_in_file_browser = False # -v, --verbosity @@ -118,17 +104,12 @@ window_size = default # --window_monitor window_monitor = 0 -# --force_window -force_window = False - # --use_projection_fill_shaders use_projection_fill_shaders = False # --use_projection_stroke_shaders use_projection_stroke_shaders = False -movie_file_extension = .mp4 - # Maximum number of partial movie files being encoded concurrently while the # scene continues rendering. 1 encodes each animation's file before the next # animation starts; values > 1 overlap encoding with rendering (bounds pipeline @@ -145,10 +126,8 @@ frame_rate = 60 pixel_height = 1080 pixel_width = 1920 -# Use -1 to set max_files_cached to infinity. +# Use -1 to keep an unlimited number of cached segment files. max_files_cached = 100 -#Flush cache will delete all the cached partial-movie-files. -flush_cache = False disable_caching = False # Disable the warning when there are too much submobjects to hash. disable_caching_warning = False @@ -197,19 +176,6 @@ col1 = col2 = epilog = -# Overrides the default output folders, NOT the output file names. Note that -# if the custom_folders flag is present, the Tex and text files will not be put -# under media_dir, as is the default. -[custom_folders] -media_dir = videos -video_dir = {media_dir} -sections_dir = {media_dir} -images_dir = {media_dir} -text_dir = {media_dir}/temp_files -tex_dir = {media_dir}/temp_files -log_dir = {media_dir}/temp_files -partial_movie_dir = {media_dir}/partial_movie_files/{scene_name} - # Rich settings [logger] logging_keyword = bold yellow @@ -228,9 +194,19 @@ log_height = -1 log_timestamps = True repr_number = green -[ffmpeg] -# Uncomment the following line to manually set the loglevel for ffmpeg. See -# ffmpeg manpage for accepted values +[video_encoder] +# Cached video-segment encoder. "auto" selects a codec and pixel format from +# the requested output format and alpha requirement. +codec = auto +pixel_format = auto + +[video_encoder.options] +# Arbitrary codec key/value options can be added here, for example: +# crf = 18 +# preset = slow + +[media] +# Logging level for media encoding, decoding, filtering, and muxing. loglevel = ERROR [jupyter] diff --git a/manim/_config/logger_utils.py b/manim/_config/logger_utils.py index 427cf41ca0..cb2a57049f 100644 --- a/manim/_config/logger_utils.py +++ b/manim/_config/logger_utils.py @@ -148,27 +148,14 @@ def parse_theme(parser: configparser.SectionProxy) -> Theme | None: return custom_theme -def set_file_logger(scene_name: str, module_name: str, log_dir: Path) -> None: - """Add a file handler to manim logger. - - The path to the file is built using ``config.log_dir``. +def set_file_logger(log_file_path: Path) -> None: + """Add a file handler for one exact, already resolved log path. Parameters ---------- - scene_name - The name of the scene, used in the name of the log file. - module_name - The name of the module, used in the name of the log file. - log_dir - Path to the folder where log files are stored. + log_file_path + Exact path of the log file for this scene. """ - # Note: The log file name will be - # _.log, gotten from config. So it - # can differ from the real name of the scene. would only - # appear if scene name was provided when manim was called. - log_file_name = f"{module_name}_{scene_name}.log" - log_file_path = log_dir / log_file_name - file_handler = logging.FileHandler(log_file_path, mode="w") file_handler.setFormatter(JSONFormatter()) diff --git a/manim/_config/output.py b/manim/_config/output.py new file mode 100644 index 0000000000..059741b4e7 --- /dev/null +++ b/manim/_config/output.py @@ -0,0 +1,113 @@ +"""Resolved output configuration for one render session.""" + +from __future__ import annotations + +__all__ = ["OutputFormat", "OutputSpec"] + +from dataclasses import dataclass +from enum import StrEnum + + +class OutputFormat(StrEnum): + """Supported primary render artifacts.""" + + NONE = "none" + AUTO = "auto" + MP4 = "mp4" + WEBM = "webm" + MOV = "mov" + GIF = "gif" + PNG = "png" + PNG_SEQUENCE = "png-sequence" + + @classmethod + def parse(cls, value: str | OutputFormat | None) -> OutputFormat: + """Normalize a compatibility config value.""" + if value is None or value == "": + return cls.AUTO + if isinstance(value, cls): + return value + if not isinstance(value, str): + raise ValueError(f"Invalid output format: {value!r}") + normalized = value.lower().replace("_", "-") + return cls(normalized) + + +_VIDEO_FORMATS = frozenset( + {OutputFormat.MP4, OutputFormat.WEBM, OutputFormat.MOV, OutputFormat.GIF} +) + + +@dataclass(frozen=True, slots=True) +class OutputSpec: + """Immutable, validated output intent for one render session. + + ``format`` is concrete: ``AUTO`` is resolved before this object is created. + ``fallback_to_still`` allows an automatically selected video format to + produce a last-frame PNG when a scene has no play calls. The extension of + cached video segments is deliberately separate from the extension of the + final artifact; GIF output, for example, uses encoded video segments before + final GIF assembly. + """ + + format: OutputFormat + transparent: bool + save_sections: bool + fallback_to_still: bool + + def __post_init__(self) -> None: + if self.format is OutputFormat.AUTO: + raise ValueError("OutputSpec requires a concrete output format.") + if self.fallback_to_still and not self.is_video: + raise ValueError("Still-image fallback requires a video output format.") + if self.transparent and self.format is OutputFormat.MP4: + raise ValueError( + "MP4 output does not support an alpha channel. Use --format=mov " + "or --format=webm for transparent video.", + ) + if self.save_sections and not self.is_video: + raise ValueError("Section output requires a video output format.") + + @property + def enabled(self) -> bool: + """Whether this session produces a primary media artifact.""" + return self.format is not OutputFormat.NONE + + @property + def is_video(self) -> bool: + """Whether the primary artifact is time-based video.""" + return self.format in _VIDEO_FORMATS + + @property + def is_still(self) -> bool: + """Whether only the last frame of the scene is written as PNG.""" + return self.format is OutputFormat.PNG + + @property + def is_image_sequence(self) -> bool: + """Whether every rendered frame is written as a PNG image.""" + return self.format is OutputFormat.PNG_SEQUENCE + + @property + def is_gif(self) -> bool: + return self.format is OutputFormat.GIF + + @property + def artifact_extension(self) -> str | None: + """Extension of the requested primary artifact.""" + if self.format is OutputFormat.NONE: + return None + if self.format in {OutputFormat.PNG, OutputFormat.PNG_SEQUENCE}: + return ".png" + return f".{self.format.value}" + + @property + def segment_extension(self) -> str: + """Container extension used for cached rendered video segments.""" + if not self.is_video: + raise ValueError("Non-video output does not use video segments.") + if self.format is OutputFormat.GIF: + return ".mov" if self.transparent else ".mp4" + extension = self.artifact_extension + assert extension is not None + return extension diff --git a/manim/_config/output_plan.py b/manim/_config/output_plan.py new file mode 100644 index 0000000000..ce07bc8a92 --- /dev/null +++ b/manim/_config/output_plan.py @@ -0,0 +1,374 @@ +"""Internal scene-output path planning.""" + +from __future__ import annotations + +import os +import re +import unicodedata +from dataclasses import dataclass +from pathlib import Path +from typing import Protocol + +from manim import __version__ + +from .output import OutputFormat, OutputSpec + + +class _LayoutConfigSource(Protocol): + input_file: str | Path + output_file: str | Path + log_to_file: bool + zero_pad: int + + def get_dir(self, key: str, **kwargs: str) -> Path | None: ... + + +@dataclass(frozen=True, slots=True) +class MediaLayoutSpec: + """Exact output directories captured for one scene.""" + + video_dir: Path | None + images_dir: Path | None + sections_dir: Path | None + partial_movie_dir: Path | None + log_dir: Path | None + zero_pad: int + + def __post_init__(self) -> None: + for path in ( + self.video_dir, + self.images_dir, + self.sections_dir, + self.partial_movie_dir, + self.log_dir, + ): + if path is not None and not path.is_absolute(): + raise ValueError("Media layout paths must be absolute.") + if not 0 <= self.zero_pad <= 9: + raise ValueError("PNG zero padding must be between 0 and 9.") + + +@dataclass(frozen=True, slots=True) +class OutputPlan: + """Exact paths and dynamic child-name policy for one scene output. + + The plan retains the immutable :class:`OutputSpec` from which it was + resolved. ``fallback_image`` is present only when automatic video output may + fall back to a last-frame PNG for a scene without play calls. For video output, + ``concat_manifest`` is the persistent diagnostic snapshot of the main scene's + segment order; assembly does not consume it. + """ + + output: OutputSpec + primary_artifact: Path | None + fallback_image: Path | None + image_sequence_dir: Path | None + segment_cache_dir: Path | None + sections_dir: Path | None + section_index: Path | None + subcaption_file: Path | None + concat_manifest: Path | None + output_stem: str + segment_extension: str | None + zero_pad: int + + def image_frame_path(self, frame_index: int) -> Path: + """Return the exact path for one PNG-sequence frame.""" + if self.image_sequence_dir is None: + raise ValueError("This output plan does not contain an image sequence.") + if frame_index < 0: + raise ValueError("Frame indices must be non-negative.") + return self.image_sequence_dir / f"{frame_index:0{self.zero_pad}d}.png" + + def segment_path(self, cache_key: str) -> Path: + """Return the exact path for one silent cached video segment.""" + if self.segment_cache_dir is None or self.segment_extension is None: + raise ValueError("This output plan does not contain video segments.") + if not cache_key or Path(cache_key).name != cache_key: + raise ValueError("A cache key must be a non-empty filename component.") + return self.segment_cache_dir / f"{cache_key}{self.segment_extension}" + + def section_path(self, index: int, name: str) -> Path: + """Return the exact path for one derived section video.""" + if self.sections_dir is None or self.segment_extension is None: + raise ValueError("This output plan does not contain section output.") + if index < 0: + raise ValueError("Section indices must be non-negative.") + section_slug = _slugify_section_name(name) + return self.sections_dir / ( + f"{self.output_stem}_{index:04}_{section_slug}{self.segment_extension}" + ) + + +def _slugify_section_name(name: str) -> str: + """Return a safe filename component while preserving Unicode words.""" + if not isinstance(name, str): + raise TypeError("Section names must be strings.") + normalized = unicodedata.normalize("NFKC", name) + return re.sub(r"[^\w]+", "-", normalized).strip("-_") or "section" + + +def _absolute_lexical(path: Path, working_directory: Path) -> Path: + if not working_directory.is_absolute(): + raise ValueError("The output planning working directory must be absolute.") + anchored = path if path.is_absolute() else working_directory / path + return Path(os.path.normpath(anchored)) + + +def _required_dir( + config: _LayoutConfigSource, + key: str, + *, + working_directory: Path, + module_name: str, + scene_name: str, +) -> Path: + path = config.get_dir(key, module_name=module_name, scene_name=scene_name) + if path is None: + raise ValueError(f"{key} must not be empty for the requested output.") + return _absolute_lexical(path, working_directory) + + +def resolve_segment_cache_directory( + config: _LayoutConfigSource, + *, + module_name: str, + scene_name: str, + working_directory: Path, +) -> Path: + """Resolve the segment-cache directory for one scene.""" + return _required_dir( + config, + "partial_movie_dir", + working_directory=working_directory, + module_name=module_name, + scene_name=scene_name, + ) + + +def resolve_module_name(config: _LayoutConfigSource) -> str: + """Resolve the source module name used by configured directory templates.""" + if not config.input_file: + return "" + input_file = config.get_dir("input_file") + if input_file is None: + return "" + return input_file.stem + + +def resolve_requested_output_name( + config: _LayoutConfigSource, +) -> Path | None: + """Resolve the optional user-requested output name without choosing a format.""" + if not config.output_file: + return None + output_file = config.get_dir("output_file") + if output_file is None: + return None + return output_file + + +def resolve_media_layout( + config: _LayoutConfigSource, + output: OutputSpec, + *, + module_name: str, + scene_name: str, + working_directory: Path, +) -> MediaLayoutSpec: + """Capture exact directories needed by one concrete scene output.""" + images_dir = None + video_dir = None + sections_dir = None + partial_movie_dir = None + + if output.is_still or output.is_image_sequence or output.fallback_to_still: + images_dir = _required_dir( + config, + "images_dir", + working_directory=working_directory, + module_name=module_name, + scene_name=scene_name, + ) + if output.is_video: + video_dir = _required_dir( + config, + "video_dir", + working_directory=working_directory, + module_name=module_name, + scene_name=scene_name, + ) + partial_movie_dir = resolve_segment_cache_directory( + config, + working_directory=working_directory, + module_name=module_name, + scene_name=scene_name, + ) + if output.save_sections: + sections_dir = _required_dir( + config, + "sections_dir", + working_directory=working_directory, + module_name=module_name, + scene_name=scene_name, + ) + + log_dir = None + if config.log_to_file: + log_dir = _required_dir( + config, + "log_dir", + working_directory=working_directory, + module_name=module_name, + scene_name=scene_name, + ) + + return MediaLayoutSpec( + video_dir=video_dir, + images_dir=images_dir, + sections_dir=sections_dir, + partial_movie_dir=partial_movie_dir, + log_dir=log_dir, + zero_pad=config.zero_pad, + ) + + +def _add_artifact_extension(path: Path, extension: str) -> Path: + if path.suffix == extension: + return path + return path.with_suffix(path.suffix + extension) + + +def _versioned(path: Path) -> Path: + return path.with_name(f"{path.stem}_ManimCE_v{__version__}{path.suffix}") + + +def _output_path(root: Path, name: Path, extension: str) -> Path: + return root / _add_artifact_extension(name, extension) + + +def resolve_output_plan( + layout: MediaLayoutSpec, + output: OutputSpec, + *, + scene_name: str, + requested_output_name: Path | None, +) -> OutputPlan: + """Resolve all stable artifact and cache paths for one scene.""" + if not scene_name: + raise ValueError("A scene name is required for output planning.") + + output_name = requested_output_name or Path(scene_name) + if output_name.name in {"", ".", ".."}: + raise ValueError("The requested output name must contain a filename.") + output_stem = output_name.stem + + if output.format is OutputFormat.NONE: + return OutputPlan( + output=output, + primary_artifact=None, + fallback_image=None, + image_sequence_dir=None, + segment_cache_dir=None, + sections_dir=None, + section_index=None, + subcaption_file=None, + concat_manifest=None, + output_stem=output_stem, + segment_extension=None, + zero_pad=layout.zero_pad, + ) + + default_name = requested_output_name is None + normalized_png = None + versioned_png = None + if output.is_still or output.is_image_sequence or output.fallback_to_still: + images_dir = layout.images_dir + if images_dir is None: + raise ValueError("Image output requires an images directory.") + normalized_png = _output_path(images_dir, output_name, ".png") + versioned_png = _versioned(normalized_png) if default_name else normalized_png + + if output.is_still: + assert versioned_png is not None + return OutputPlan( + output=output, + primary_artifact=versioned_png, + fallback_image=None, + image_sequence_dir=None, + segment_cache_dir=None, + sections_dir=None, + section_index=None, + subcaption_file=None, + concat_manifest=None, + output_stem=output_stem, + segment_extension=None, + zero_pad=layout.zero_pad, + ) + + if output.is_image_sequence: + assert normalized_png is not None + sequence_dir = normalized_png.with_suffix("") + return OutputPlan( + output=output, + primary_artifact=sequence_dir, + fallback_image=None, + image_sequence_dir=sequence_dir, + segment_cache_dir=None, + sections_dir=None, + section_index=None, + subcaption_file=None, + concat_manifest=None, + output_stem=output_stem, + segment_extension=None, + zero_pad=layout.zero_pad, + ) + + if not output.is_video: + raise ValueError(f"Unsupported output format: {output.format.value}") + if layout.video_dir is None or layout.partial_movie_dir is None: + raise ValueError("Video output requires video and segment-cache directories.") + + artifact_extension = output.artifact_extension + assert artifact_extension is not None + primary_artifact = _output_path( + layout.video_dir, + output_name, + artifact_extension, + ) + if output.is_gif and default_name: + primary_artifact = _versioned(primary_artifact) + + sections_dir = layout.sections_dir + if output.save_sections and sections_dir is None: + raise ValueError("Section output requires a sections directory.") + section_index = ( + sections_dir / f"{output_stem}.json" if sections_dir is not None else None + ) + + return OutputPlan( + output=output, + primary_artifact=primary_artifact, + fallback_image=versioned_png if output.fallback_to_still else None, + image_sequence_dir=None, + segment_cache_dir=layout.partial_movie_dir, + sections_dir=sections_dir, + section_index=section_index, + subcaption_file=primary_artifact.with_suffix(".srt"), + concat_manifest=layout.partial_movie_dir / "partial_movie_file_list.txt", + output_stem=output_stem, + segment_extension=output.segment_extension, + zero_pad=layout.zero_pad, + ) + + +def resolve_file_log_path( + layout: MediaLayoutSpec, + *, + module_name: str, + scene_name: str, +) -> Path | None: + """Return the exact optional log-file path for one scene.""" + if layout.log_dir is None: + return None + return layout.log_dir / f"{module_name}_{scene_name}.log" diff --git a/manim/_config/render_session.py b/manim/_config/render_session.py new file mode 100644 index 0000000000..f615d7ef08 --- /dev/null +++ b/manim/_config/render_session.py @@ -0,0 +1,126 @@ +"""Resolved render-session configuration.""" + +from __future__ import annotations + +__all__ = [ + "PresentationSpec", + "RenderSessionSpec", + "resolve_render_session", +] + +from dataclasses import dataclass +from typing import Protocol + +from manim.renderer.protocol import RendererCapabilities + +from .output import OutputFormat, OutputSpec +from .video_encoder import VideoEncoderSpec, resolve_video_encoder + + +@dataclass(frozen=True, slots=True) +class PresentationSpec: + """Immutable presentation requests for one render session.""" + + open_after_render: bool + live_preview: bool + show_in_file_browser: bool + + +@dataclass(frozen=True, slots=True) +class RenderSessionSpec: + """Validated artifact, presentation, and execution intent for one session.""" + + output: OutputSpec + presentation: PresentationSpec + dry_run: bool + video_encoder: VideoEncoderSpec | None + + +class _SessionConfigSource(Protocol): + format: str | OutputFormat | None + save_sections: bool + transparent: bool + preview: bool + live_preview: bool + show_in_file_browser: bool + enable_gui: bool + dry_run: bool + pixel_width: int + pixel_height: int + frame_rate: float + video_codec: str + pixel_format: str + video_encoder_options: dict[str, str] + + +def resolve_render_session( + config: _SessionConfigSource, + capabilities: RendererCapabilities, + *, + renderer_name: str, +) -> RenderSessionSpec: + """Resolve and validate one renderer-independent session request.""" + live_preview = config.live_preview or config.enable_gui + dry_run = config.dry_run + requested_format = OutputFormat.parse(config.format) + fallback_to_still = False + if dry_run: + requested_format = OutputFormat.NONE + save_sections = False + else: + save_sections = config.save_sections + if requested_format is OutputFormat.AUTO: + if live_preview: + requested_format = OutputFormat.NONE + else: + requested_format = ( + OutputFormat.MOV if config.transparent else OutputFormat.MP4 + ) + fallback_to_still = True + + output = OutputSpec( + format=requested_format, + transparent=config.transparent, + save_sections=save_sections, + fallback_to_still=fallback_to_still, + ) + presentation = PresentationSpec( + open_after_render=config.preview, + live_preview=live_preview, + show_in_file_browser=config.show_in_file_browser, + ) + + if live_preview and not capabilities.live_preview: + raise ValueError( + f"{renderer_name} does not support live preview. " + "Select a renderer with live-preview support or remove --live-preview.", + ) + if live_preview and dry_run: + raise ValueError("--live-preview cannot be combined with --dry_run.") + if live_preview and output.is_still: + raise ValueError( + "Live preview cannot be combined with last-frame PNG output.", + ) + if presentation.open_after_render and not output.enabled: + raise ValueError( + "--preview requires a media artifact. Choose a concrete --format when " + "using --live-preview.", + ) + if presentation.show_in_file_browser and not output.enabled: + raise ValueError("--show_in_file_browser requires a media artifact.") + + video_encoder = resolve_video_encoder( + output, + width=config.pixel_width, + height=config.pixel_height, + frame_rate=config.frame_rate, + codec=config.video_codec, + pixel_format=config.pixel_format, + options=config.video_encoder_options, + ) + return RenderSessionSpec( + output=output, + presentation=presentation, + dry_run=dry_run, + video_encoder=video_encoder, + ) diff --git a/manim/_config/utils.py b/manim/_config/utils.py index 8d1de7a9d4..9176e41b1d 100644 --- a/manim/_config/utils.py +++ b/manim/_config/utils.py @@ -27,6 +27,7 @@ import numpy as np from manim import constants +from manim._config.output import OutputFormat from manim.constants import RendererType from manim.utils.color import ManimColor from manim.utils.tex import TexTemplate @@ -262,15 +263,12 @@ class MyScene(Scene): ... "assets_dir", "background_color", "background_opacity", - "custom_folders", "disable_caching", "disable_caching_warning", "dry_run", "encoder_queue_size", "enable_wireframe", - "ffmpeg_loglevel", "format", - "flush_cache", "frame_height", "frame_rate", "frame_width", @@ -280,26 +278,25 @@ class MyScene(Scene): ... "images_dir", "input_file", "media_embed", + "media_loglevel", "media_width", "log_dir", "log_to_file", "max_files_cached", "max_inflight_encoders", "media_dir", - "movie_file_extension", "notify_outdated_version", "output_file", "partial_movie_dir", + "pixel_format", "pixel_height", "pixel_width", "plugins", "preview", + "live_preview", "progress_bar", "quality", - "save_as_gif", "save_sections", - "save_last_frame", - "save_pngs", "scene_names", "seed", "show_in_file_browser", @@ -314,16 +311,16 @@ class MyScene(Scene): ... "use_projection_fill_shaders", "use_projection_stroke_shaders", "verbosity", + "video_codec", "video_dir", + "video_encoder_options", "sections_dir", "fullscreen", "window_position", "window_size", "window_monitor", "write_all", - "write_to_movie", "zero_pad", - "force_window", "no_latex_cleanup", "preview_command", } @@ -580,28 +577,37 @@ def digest_parser(self, parser: configparser.ConfigParser) -> Self: """ self._parser = parser + self.format = parser["CLI"].get("format", fallback="auto", raw=True) + self.video_codec = parser["video_encoder"].get( + "codec", + fallback="auto", + raw=True, + ) + self.pixel_format = parser["video_encoder"].get( + "pixel_format", + fallback="auto", + raw=True, + ) + self.video_encoder_options = dict( + parser.items("video_encoder.options", raw=True), + ) + # boolean keys for key in [ "notify_outdated_version", - "write_to_movie", - "save_last_frame", "write_all", - "save_pngs", - "save_as_gif", "save_sections", "preview", + "live_preview", "show_in_file_browser", "log_to_file", "disable_caching", "disable_caching_warning", - "flush_cache", - "custom_folders", "enable_gui", "fullscreen", "use_projection_fill_shaders", "use_projection_stroke_shaders", "enable_wireframe", - "force_window", "no_latex_cleanup", "dry_run", ]: @@ -637,7 +643,6 @@ def digest_parser(self, parser: configparser.ConfigParser) -> Self: "partial_movie_dir", "input_file", "output_file", - "movie_file_extension", "background_color", "renderer", "window_position", @@ -691,9 +696,9 @@ def digest_parser(self, parser: configparser.ConfigParser) -> Self: if progress_bar: self.progress_bar = progress_bar - ffmpeg_loglevel = parser["ffmpeg"].get("loglevel") - if ffmpeg_loglevel: - self.ffmpeg_loglevel = ffmpeg_loglevel + media_loglevel = parser["media"].get("loglevel") + if media_loglevel: + self.media_loglevel = media_loglevel try: media_embed = parser["jupyter"].getboolean("media_embed") @@ -755,21 +760,19 @@ def digest_args(self, args: argparse.Namespace) -> Self: self.input_file = Path(args.file).absolute() self.scene_names = args.scene_names if args.scene_names is not None else [] - self.output_file = args.output_file + if args.output_file is not None: + self.output_file = args.output_file for key in [ "notify_outdated_version", "preview", + "live_preview", "show_in_file_browser", - "write_to_movie", "save_last_frame", - "save_pngs", - "save_as_gif", "save_sections", "write_all", "disable_caching", "format", - "flush_cache", "progress_bar", "transparent", "scene_names", @@ -782,13 +785,15 @@ def digest_args(self, args: argparse.Namespace) -> Self: "use_projection_stroke_shaders", "zero_pad", "enable_wireframe", - "force_window", "dry_run", "no_latex_cleanup", "preview_command", "seed", "max_inflight_encoders", "encoder_queue_size", + "video_codec", + "pixel_format", + "video_encoder_options", ]: if hasattr(args, key): attr = getattr(args, key) @@ -809,9 +814,6 @@ def digest_args(self, args: argparse.Namespace) -> Self: if attr is not None: self[key] = attr - if self["save_last_frame"]: - self["write_to_movie"] = False - # Handle the -n flag. nflag = args.from_animation_number if nflag: @@ -836,31 +838,10 @@ def digest_args(self, args: argparse.Namespace) -> Self: if fps: self.frame_rate = float(fps) - # Handle --custom_folders - if args.custom_folders: - for opt in [ - "media_dir", - "video_dir", - "sections_dir", - "images_dir", - "text_dir", - "tex_dir", - "log_dir", - "partial_movie_dir", - ]: - self[opt] = self._parser["custom_folders"].get(opt, raw=True) - # --media_dir overrides the default.cfg file - if hasattr(args, "media_dir") and args.media_dir: - self.media_dir = args.media_dir - # Handle --tex_template if args.tex_template: self.tex_template = TexTemplate.from_file(args.tex_template) - if self.renderer == RendererType.OPENGL and args.write_to_movie is None: - # --write_to_movie was not passed on the command line, so don't generate video. - self["write_to_movie"] = False - # Handle --gui_location flag. if args.gui_location is not None: self.gui_location = args.gui_location @@ -911,13 +892,22 @@ def digest_file(self, filename: StrPath) -> Self: @property def preview(self) -> bool: - """Whether to play the rendered movie (-p).""" - return self._d["preview"] or self._d["enable_gui"] + """Whether to open the completed artifact after rendering (-p).""" + return self._d["preview"] @preview.setter def preview(self, value: bool) -> None: self._set_boolean("preview", value) + @property + def live_preview(self) -> bool: + """Whether to display frames in a renderer-provided live preview (-l).""" + return self._d["live_preview"] + + @live_preview.setter + def live_preview(self, value: bool) -> None: + self._set_boolean("live_preview", value) + @property def show_in_file_browser(self) -> bool: """Whether to show the output file in the file browser (-f).""" @@ -954,23 +944,19 @@ def notify_outdated_version(self) -> bool: def notify_outdated_version(self, value: bool) -> None: self._set_boolean("notify_outdated_version", value) - @property - def write_to_movie(self) -> bool: - """Whether to render the scene to a movie file (-w).""" - return self._d["write_to_movie"] - - @write_to_movie.setter - def write_to_movie(self, value: bool) -> None: - self._set_boolean("write_to_movie", value) - @property def save_last_frame(self) -> bool: - """Whether to save the last frame of the scene as an image file (-s).""" - return self._d["save_last_frame"] + """Whether to save the last frame of the scene as a PNG (-s).""" + return OutputFormat.parse(self.format) is OutputFormat.PNG @save_last_frame.setter def save_last_frame(self, value: bool) -> None: - self._set_boolean("save_last_frame", value) + if not isinstance(value, bool): + raise ValueError("save_last_frame must be boolean") + if value: + self.format = OutputFormat.PNG + elif self.save_last_frame: + self.format = OutputFormat.AUTO @property def write_all(self) -> bool: @@ -981,24 +967,6 @@ def write_all(self) -> bool: def write_all(self, value: bool) -> None: self._set_boolean("write_all", value) - @property - def save_pngs(self) -> bool: - """Whether to save all frames in the scene as images files (-g).""" - return self._d["save_pngs"] - - @save_pngs.setter - def save_pngs(self, value: bool) -> None: - self._set_boolean("save_pngs", value) - - @property - def save_as_gif(self) -> bool: - """Whether to save the rendered scene in .gif format (-i).""" - return self._d["save_as_gif"] - - @save_as_gif.setter - def save_as_gif(self, value: bool) -> None: - self._set_boolean("save_as_gif", value) - @property def save_sections(self) -> bool: """Whether to save single videos for each section in addition to the movie file.""" @@ -1017,15 +985,6 @@ def enable_wireframe(self) -> bool: def enable_wireframe(self, value: bool) -> None: self._set_boolean("enable_wireframe", value) - @property - def force_window(self) -> bool: - """Whether to force window when using the opengl renderer.""" - return self._d["force_window"] - - @force_window.setter - def force_window(self, value: bool) -> None: - self._set_boolean("force_window", value) - @property def no_latex_cleanup(self) -> bool: """Prevents deletion of .aux, .dvi, and .log files produced by Tex and MathTex.""" @@ -1059,35 +1018,73 @@ def verbosity(self, val: str) -> None: @property def format(self) -> str | None: - """File format; "png", "gif", "mp4", "webm" or "mov".""" + """Primary output format. + + ``png`` writes only the last frame of the scene; + ``png-sequence`` writes every rendered frame. ``auto`` selects MP4 for + opaque output and MOV for transparent output, while ``none`` disables + media output. + """ return self._d["format"] @format.setter # noqa: A003 - def format(self, val: str) -> None: - self._set_from_list( - "format", - val, - [None, "png", "gif", "mp4", "mov", "webm"], - ) - self.resolve_movie_file_extension(self.transparent) - if self.format == "webm": + def format(self, val: str | OutputFormat | None) -> None: + output_format = OutputFormat.parse(val) + self._d["format"] = output_format.value + if output_format is OutputFormat.WEBM: logger.warning( "Output format set as webm, this can be slower than other formats", ) @property - def ffmpeg_loglevel(self) -> str: - """Verbosity level of ffmpeg (no flag).""" - return self._d["ffmpeg_loglevel"] + def video_codec(self) -> str: + """Video encoder used for cached segments.""" + return self._d["video_codec"] + + @video_codec.setter + def video_codec(self, value: str) -> None: + if not isinstance(value, str) or not value: + raise ValueError("video_codec must be a non-empty string") + self._d["video_codec"] = value + + @property + def pixel_format(self) -> str: + """Pixel format used for cached video segments.""" + return self._d["pixel_format"] + + @pixel_format.setter + def pixel_format(self, value: str) -> None: + if not isinstance(value, str) or not value: + raise ValueError("pixel_format must be a non-empty string") + self._d["pixel_format"] = value + + @property + def video_encoder_options(self) -> dict[str, str]: + """Codec options used for cached video segments.""" + return dict(self._d["video_encoder_options"]) - @ffmpeg_loglevel.setter - def ffmpeg_loglevel(self, val: str) -> None: + @video_encoder_options.setter + def video_encoder_options(self, value: Mapping[str, str]) -> None: + if not isinstance(value, Mapping) or any( + not isinstance(key, str) or not isinstance(option, str) + for key, option in value.items() + ): + raise TypeError("video_encoder_options must map strings to strings") + self._d["video_encoder_options"] = dict(value) + + @property + def media_loglevel(self) -> str: + """Logging level for media operations.""" + return self._d["media_loglevel"] + + @media_loglevel.setter + def media_loglevel(self, value: str) -> None: self._set_from_list( - "ffmpeg_loglevel", - val, + "media_loglevel", + value, ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"], ) - logging.getLogger("libav").setLevel(self.ffmpeg_loglevel) + logging.getLogger("libav").setLevel(self.media_loglevel) @property def media_embed(self) -> bool | None: @@ -1236,7 +1233,12 @@ def max_files_cached(self) -> int: @max_files_cached.setter def max_files_cached(self, value: int) -> None: - self._set_pos_number("max_files_cached", value, True) + if isinstance(value, int) and value >= -1: + self._d["max_files_cached"] = value + else: + raise ValueError( + "max_files_cached must be a non-negative integer or -1 for unlimited", + ) @property def max_inflight_encoders(self) -> int: @@ -1278,15 +1280,6 @@ def window_monitor(self) -> int: def window_monitor(self, value: int) -> None: self._set_pos_number("window_monitor", value, True) - @property - def flush_cache(self) -> bool: - """Whether to delete all the cached partial movie files.""" - return self._d["flush_cache"] - - @flush_cache.setter - def flush_cache(self, value: bool) -> None: - self._set_boolean("flush_cache", value) - @property def disable_caching(self) -> bool: """Whether to use scene caching.""" @@ -1305,15 +1298,6 @@ def disable_caching_warning(self) -> bool: def disable_caching_warning(self, value: bool) -> None: self._set_boolean("disable_caching_warning", value) - @property - def movie_file_extension(self) -> str: - """Either .mp4, .webm or .mov.""" - return self._d["movie_file_extension"] - - @movie_file_extension.setter - def movie_file_extension(self, value: str) -> None: - self._set_from_list("movie_file_extension", value, [".mp4", ".mov", ".webm"]) - @property def background_opacity(self) -> float: """A number between 0.0 (fully transparent) and 1.0 (fully opaque).""" @@ -1322,8 +1306,6 @@ def background_opacity(self) -> float: @background_opacity.setter def background_opacity(self, value: float) -> None: self._set_between("background_opacity", value, 0, 1) - if self.background_opacity < 1: - self.resolve_movie_file_extension(is_transparent=True) @property def frame_size(self) -> tuple[int, int]: @@ -1365,7 +1347,6 @@ def transparent(self) -> bool: @transparent.setter def transparent(self, value: bool) -> None: self._d["background_opacity"] = float(not value) - self.resolve_movie_file_extension(value) @property def dry_run(self) -> bool: @@ -1374,12 +1355,7 @@ def dry_run(self) -> bool: @dry_run.setter def dry_run(self, val: bool) -> None: - self._d["dry_run"] = val - if val: - self.write_to_movie = False - self.write_all = False - self.save_last_frame = False - self.format = None + self._set_boolean("dry_run", val) @property def renderer(self) -> RendererType: @@ -1477,22 +1453,6 @@ def window_size(self) -> str | tuple[int, ...]: def window_size(self, value: str | tuple[int, ...]) -> None: self._d.__setitem__("window_size", value) - def resolve_movie_file_extension(self, is_transparent: bool) -> None: - prev_file_extension = self.movie_file_extension - if is_transparent: - self.movie_file_extension = ".webm" if self.format == "webm" else ".mov" - elif self.format == "webm": - self.movie_file_extension = ".webm" - elif self.format == "mov": - self.movie_file_extension = ".mov" - else: - self.movie_file_extension = ".mp4" - if self.movie_file_extension != prev_file_extension: - logger.warning( - f"Output format changed to '{self.movie_file_extension}' " - "to support transparency", - ) - @property def enable_gui(self) -> bool: """Enable GUI interaction.""" @@ -1777,15 +1737,6 @@ def partial_movie_dir(self) -> str: def partial_movie_dir(self, value: str | Path) -> None: self._set_dir("partial_movie_dir", value) - @property - def custom_folders(self) -> str: - """Whether to use custom folder output.""" - return self._d["custom_folders"] - - @custom_folders.setter - def custom_folders(self, value: str | Path) -> None: - self._set_dir("custom_folders", value) - @property def input_file(self) -> str | Path: """Input file name.""" diff --git a/manim/_config/video_encoder.py b/manim/_config/video_encoder.py new file mode 100644 index 0000000000..496bdea225 --- /dev/null +++ b/manim/_config/video_encoder.py @@ -0,0 +1,235 @@ +"""Resolved settings for cached video-segment encoding.""" + +from __future__ import annotations + +import hashlib +import json +import math +from collections.abc import Mapping +from dataclasses import dataclass +from fractions import Fraction + +import av + +from .output import OutputSpec + +__all__ = [ + "VideoEncoderSpec", + "resolve_video_encoder", + "to_av_frame_rate", + "video_encoder_fingerprint", +] + +_CONFLICTING_OPTION_KEYS = frozenset( + { + "codec", + "codec_name", + "framerate", + "height", + "pix_fmt", + "pixel_format", + "rate", + "video_size", + "width", + }, +) +_DEFAULT_OPTIONS = { + "libx264": {"crf": "23"}, + "libvpx-vp9": {"crf": "23"}, + "qtrle": {}, +} + + +@dataclass(frozen=True, slots=True) +class VideoEncoderSpec: + """Complete byte-affecting settings for one cached video segment.""" + + container_format: str + codec: str + pixel_format: str + width: int + height: int + frame_rate: Fraction + options: tuple[tuple[str, str], ...] + + +def video_encoder_fingerprint(spec: VideoEncoderSpec | None) -> str: + """Return the stable cache-identity token for resolved encoder settings.""" + if spec is None: + return "none" + payload = { + "schema": "manim-video-segment-v1", + "container_format": spec.container_format, + "codec": spec.codec, + "pixel_format": spec.pixel_format, + "width": spec.width, + "height": spec.height, + "frame_rate": { + "numerator": spec.frame_rate.numerator, + "denominator": spec.frame_rate.denominator, + }, + "options": [ + {"name": name, "value": value} for name, value in sorted(spec.options) + ], + } + serialized = json.dumps( + payload, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ) + return hashlib.sha256(serialized.encode("utf-8")).hexdigest()[:16] + + +def to_av_frame_rate(frame_rate: int | float | Fraction) -> Fraction: + """Return a positive exact rate suitable for a PyAV video stream.""" + if isinstance(frame_rate, bool): + raise ValueError("frame rate must be a positive finite number") + if isinstance(frame_rate, Fraction): + resolved = frame_rate + elif isinstance(frame_rate, int): + resolved = Fraction(frame_rate, 1) + elif isinstance(frame_rate, float) and math.isfinite(frame_rate): + if abs(frame_rate - round(frame_rate)) < 1e-4: + resolved = Fraction(round(frame_rate), 1) + else: + ntsc_rate = Fraction(round(frame_rate * 1001 / 1000) * 1000, 1001) + resolved = ( + ntsc_rate + if abs(frame_rate - float(ntsc_rate)) < 0.02 + else Fraction(str(frame_rate)) + ) + else: + raise ValueError("frame rate must be a positive finite number") + + if resolved <= 0: + raise ValueError("frame rate must be positive") + return resolved + + +def _default_profile(output: OutputSpec) -> tuple[str, str, str]: + extension = output.segment_extension + if extension is None: + raise ValueError("Video output requires a segment container.") + + container_format = extension.removeprefix(".") + if extension == ".webm": + return ( + container_format, + "libvpx-vp9", + "yuva420p" if output.transparent else "yuv420p", + ) + if output.transparent: + return container_format, "qtrle", "argb" + return container_format, "libx264", "yuv420p" + + +def _validate_geometry(width: int, height: int) -> None: + if ( + isinstance(width, bool) + or isinstance(height, bool) + or not isinstance(width, int) + or not isinstance(height, int) + or width <= 0 + or height <= 0 + ): + raise ValueError("Video dimensions must be positive integers.") + + +def _validate_profile( + *, + container_format: str, + codec_name: str, + pixel_format: str, + transparent: bool, +) -> None: + try: + container = av.format.ContainerFormat(container_format) + except ValueError as error: + raise ValueError(f"Unknown output container: {container_format}") from error + if not container.is_output: + raise ValueError(f"Container does not support output: {container_format}") + + try: + codec = av.codec.Codec(codec_name, "w") + except ValueError as error: + raise ValueError(f"Unknown video encoder: {codec_name}") from error + if codec.type != "video": + raise ValueError(f"Encoder is not a video encoder: {codec_name}") + + try: + video_format = av.VideoFormat(pixel_format) + except ValueError as error: + raise ValueError(f"Unknown pixel format: {pixel_format}") from error + + supported_formats = codec.video_formats + if supported_formats is not None and pixel_format not in { + supported.name for supported in supported_formats + }: + raise ValueError( + f"Pixel format {pixel_format} is not supported by encoder {codec_name}.", + ) + if transparent and not any( + component.is_alpha for component in video_format.components + ): + raise ValueError( + f"Transparent output requires an alpha-bearing pixel format; " + f"got {pixel_format}.", + ) + + +def resolve_video_encoder( + output: OutputSpec, + *, + width: int, + height: int, + frame_rate: int | float | Fraction, + codec: str = "auto", + pixel_format: str = "auto", + options: Mapping[str, str] | None = None, +) -> VideoEncoderSpec | None: + """Resolve and validate cached-segment encoder settings.""" + if not output.is_video: + return None + + _validate_geometry(width, height) + resolved_rate = to_av_frame_rate(frame_rate) + container_format, default_codec, default_pixel_format = _default_profile(output) + resolved_codec = default_codec if codec == "auto" else codec + resolved_pixel_format = ( + default_pixel_format if pixel_format == "auto" else pixel_format + ) + + supplied_options = {} if options is None else dict(options) + if any( + not isinstance(key, str) or not isinstance(value, str) + for key, value in supplied_options.items() + ): + raise TypeError("Video encoder option keys and values must be strings.") + conflicting = sorted(_CONFLICTING_OPTION_KEYS.intersection(supplied_options)) + if conflicting: + raise ValueError( + "Video encoder options conflict with explicit stream settings: " + + ", ".join(conflicting), + ) + + resolved_options = ( + dict(_DEFAULT_OPTIONS[default_codec]) if resolved_codec == default_codec else {} + ) + resolved_options.update(supplied_options) + + _validate_profile( + container_format=container_format, + codec_name=resolved_codec, + pixel_format=resolved_pixel_format, + transparent=output.transparent, + ) + return VideoEncoderSpec( + container_format=container_format, + codec=resolved_codec, + pixel_format=resolved_pixel_format, + width=width, + height=height, + frame_rate=resolved_rate, + options=tuple(sorted(resolved_options.items())), + ) diff --git a/manim/cli/cache/__init__.py b/manim/cli/cache/__init__.py new file mode 100644 index 0000000000..7e8f15d8bb --- /dev/null +++ b/manim/cli/cache/__init__.py @@ -0,0 +1 @@ +"""Commands for maintaining Manim's segment cache.""" diff --git a/manim/cli/cache/commands.py b/manim/cli/cache/commands.py new file mode 100644 index 0000000000..f345ea6868 --- /dev/null +++ b/manim/cli/cache/commands.py @@ -0,0 +1,122 @@ +"""Commands for maintaining cached video segments.""" + +from __future__ import annotations + +from pathlib import Path + +import cloup + +from manim._config import config, console +from manim._config.output_plan import ( + resolve_module_name, + resolve_segment_cache_directory, +) +from manim._config.utils import _determine_quality +from manim.cli.render.render_options import validate_resolution +from manim.constants import EPILOG, QUALITIES +from manim.utils.caching import clear_segment_cache + +__all__ = ["cache"] + + +@cloup.group( + context_settings=None, + no_args_is_help=True, + epilog=EPILOG, +) +def cache() -> None: + """Maintain cached video segments.""" + + +@cache.command( + context_settings=None, + no_args_is_help=True, + epilog=EPILOG, +) +@cloup.argument( + "file", + type=cloup.Path(path_type=Path, exists=True, dir_okay=False), + required=True, +) +@cloup.argument("scene_names", required=True, nargs=-1) +@cloup.option( + "-c", + "--config-file", + type=cloup.Path(path_type=Path, exists=True, dir_okay=False), + default=None, + help="Use the specified configuration file.", +) +@cloup.option( + "--media-dir", + type=cloup.Path(path_type=Path), + default=None, + help="Override the directory containing rendered media and caches.", +) +@cloup.option( + "-q", + "--quality", + type=cloup.Choice( + list(reversed([q["flag"] for q in QUALITIES.values() if q["flag"]])), + case_sensitive=False, + ), + default=None, + help="Resolve the cache directory for this render quality.", +) +@cloup.option( + "-r", + "--resolution", + callback=validate_resolution, + default=None, + help='Resolve the cache directory for resolution "W,H".', +) +@cloup.option( + "--fps", + "--frame-rate", + "frame_rate", + type=float, + default=None, + help="Resolve the cache directory for this frame rate.", +) +def clear( + *, + file: Path, + scene_names: tuple[str, ...], + config_file: Path | None, + media_dir: Path | None, + quality: str | None, + resolution: tuple[int, int] | None, + frame_rate: float | None, +) -> None: + """Delete cached segments for SCENE(S) from FILE.""" + command_config = config.copy() + selected_config_file = file if file.suffix == ".cfg" else config_file + if selected_config_file is not None: + command_config.digest_file(selected_config_file) + + if not command_config.input_file: + if file.suffix == ".cfg": + raise ValueError("A configuration file must define input_file.") + command_config.input_file = file.absolute() + + if media_dir is not None: + command_config.media_dir = media_dir + if quality is not None: + command_config.quality = _determine_quality(quality) + if resolution is not None: + command_config.frame_size = resolution + if frame_rate is not None: + command_config.frame_rate = frame_rate + + module_name = resolve_module_name(command_config) + working_directory = Path.cwd() + for scene_name in scene_names: + directory = resolve_segment_cache_directory( + command_config, + module_name=module_name, + scene_name=scene_name, + working_directory=working_directory, + ) + removed = clear_segment_cache(directory) + console.print( + f"Removed {removed} cached segment(s) for {scene_name} from {directory}.", + ) diff --git a/manim/cli/render/commands.py b/manim/cli/render/commands.py index 5c36b41a94..063c13dcaa 100644 --- a/manim/cli/render/commands.py +++ b/manim/cli/render/commands.py @@ -58,6 +58,14 @@ def __repr__(self) -> str: return str(self.__dict__) +def _validate_scene_batch_output_name(scene_classes: list[type]) -> None: + if config.output_file and (config.write_all or len(scene_classes) != 1): + raise ValueError( + "--output_file can only be used when rendering exactly one scene. " + "Remove --write_all or select a single scene.", + ) + + @cloup.command( context_settings=None, no_args_is_help=True, @@ -76,33 +84,23 @@ def render(**kwargs: Any) -> ClickArgs | dict[str, Any]: SCENES is an optional list of scenes in the file. """ - if kwargs["save_as_gif"]: - logger.warning("--save_as_gif is deprecated, please use --format=gif instead!") - kwargs["format"] = "gif" - - if kwargs["save_pngs"]: - logger.warning("--save_pngs is deprecated, please use --format=png instead!") - kwargs["format"] = "png" - - if kwargs["show_in_file_browser"]: - logger.warning( - "The short form of show_in_file_browser is deprecated and will be moved to support --format.", - ) - click_args = ClickArgs(kwargs) if kwargs["jupyter"]: return click_args config.digest_args(click_args) file = Path(config.input_file) - if config.renderer == RendererType.OPENGL: - from manim.renderer.opengl_renderer import OpenGLRenderer + try: + scene_classes = scene_classes_from_file(file) + _validate_scene_batch_output_name(scene_classes) + + if config.renderer == RendererType.OPENGL: + from manim.renderer.opengl_renderer import OpenGLRenderer - try: renderer = OpenGLRenderer() keep_running = True while keep_running: - for SceneClass in scene_classes_from_file(file): + for SceneClass in scene_classes: with tempconfig({}): scene = SceneClass(renderer) # Attach explicitly, but preserve custom Scene.render overrides. @@ -111,26 +109,20 @@ def render(**kwargs: Any) -> ClickArgs | dict[str, Any]: if rerun or config["write_all"]: renderer.num_plays = 0 continue - else: - keep_running = False - break + keep_running = False + break if config["write_all"]: keep_running = False - - except Exception: - error_console.print_exception() - sys.exit(1) - else: - for SceneClass in scene_classes_from_file(file): - try: + else: + for SceneClass in scene_classes: with tempconfig({}): scene = SceneClass() # Attach explicitly, but preserve custom Scene.render overrides. Manager(scene) scene.render() - except Exception: - error_console.print_exception() - sys.exit(1) + except Exception: + error_console.print_exception() + sys.exit(1) if config.notify_outdated_version: manim_info_url = "https://pypi.org/pypi/manim/json" diff --git a/manim/cli/render/ease_of_access_options.py b/manim/cli/render/ease_of_access_options.py index ac5e78f6a7..4a5b4ab51f 100644 --- a/manim/cli/render/ease_of_access_options.py +++ b/manim/cli/render/ease_of_access_options.py @@ -20,13 +20,18 @@ "-p", "--preview", is_flag=True, - help="Preview the Scene's animation. OpenGL does a live preview in a " - "popup window. Cairo opens the rendered video file in the system " - "default media player.", + help="Open the completed media artifact after rendering.", + default=None, + ), + option( + "-l", + "--live-preview", + is_flag=True, + help="Display frames in a renderer-provided live preview. With " + "--format=auto, no media file is written.", default=None, ), option( - "-f", "--show_in_file_browser", is_flag=True, help="Show the output file in the file browser.", diff --git a/manim/cli/render/global_options.py b/manim/cli/render/global_options.py index fc299d204a..597deb9570 100644 --- a/manim/cli/render/global_options.py +++ b/manim/cli/render/global_options.py @@ -61,25 +61,12 @@ def validate_gui_location( help="Specify the configuration file to use for render settings.", default=None, ), - option( - "--custom_folders", - is_flag=True, - default=None, - help="Use the folders defined in the [custom_folders] section of the " - "config file to define the output folder structure.", - ), option( "--disable_caching", is_flag=True, default=None, help="Disable the use of the cache (still generates cache files).", ), - option( - "--flush_cache", - is_flag=True, - help="Remove cached partial movie files.", - default=None, - ), option("--tex_template", help="Specify a custom TeX template file.", default=None), option( "-v", @@ -88,7 +75,7 @@ def validate_gui_location( ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"], case_sensitive=False, ), - help="Verbosity of CLI output. Changes ffmpeg log level unless 5+.", + help="Verbosity of CLI output. Changes media log level unless 5+.", default=None, ), option( @@ -121,12 +108,6 @@ def validate_gui_location( help="Enable wireframe debugging mode in opengl.", default=None, ), - option( - "--force_window", - is_flag=True, - help="Force window to open when using the opengl renderer, intended for debugging as it may impact performance", - default=None, - ), option( "--dry_run", is_flag=True, diff --git a/manim/cli/render/output_options.py b/manim/cli/render/output_options.py index a7613f1565..7c2bb7095f 100644 --- a/manim/cli/render/output_options.py +++ b/manim/cli/render/output_options.py @@ -20,12 +20,6 @@ default=None, help="Zero padding for PNG file names.", ), - option( - "--write_to_movie", - is_flag=True, - default=None, - help="Write the video rendered with opengl to a file.", - ), option( "--media_dir", type=Path(), diff --git a/manim/cli/render/render_options.py b/manim/cli/render/render_options.py index 4e42c134b8..a48d5038b8 100644 --- a/manim/cli/render/render_options.py +++ b/manim/cli/render/render_options.py @@ -5,6 +5,7 @@ import sys from typing import TYPE_CHECKING +from click import BadParameter from cloup import Choice, IntRange, option, option_group from manim.constants import QUALITIES, RendererType @@ -63,6 +64,35 @@ def validate_scene_range( return start, end +def validate_encoder_options( + ctx: Context, + param: Option, + value: tuple[str, ...], +) -> dict[str, str] | None: + """Parse repeatable ``KEY=VALUE`` encoder options.""" + if not value: + return None + + options: dict[str, str] = {} + for entry in value: + key, separator, option_value = entry.partition("=") + key = key.strip() + if not separator or not key or not option_value: + raise BadParameter( + "encoder options must use KEY=VALUE with nonempty key and value", + ctx=ctx, + param=param, + ) + if key in options: + raise BadParameter( + f"encoder option {key!r} was supplied more than once", + ctx=ctx, + param=param, + ) + options[key] = option_value + return options + + def validate_resolution( ctx: Context, param: Option, value: str | None ) -> tuple[int, int] | None: @@ -121,15 +151,30 @@ def validate_resolution( ), option( "--format", - type=Choice(["png", "gif", "mp4", "webm", "mov"], case_sensitive=False), + type=Choice( + [ + "auto", + "none", + "png", + "png-sequence", + "gif", + "mp4", + "webm", + "mov", + ], + case_sensitive=False, + ), default=None, + help="Primary output format. PNG renders only the last frame; " + "png-sequence writes every rendered frame.", ), option( "-s", "--save_last_frame", default=None, is_flag=True, - help="Render and save only the last frame of a scene as a PNG image.", + help="Fast-forward animations and save the last frame as PNG " + "(equivalent to --format=png).", ), option( "-q", @@ -165,6 +210,27 @@ def validate_resolution( default=None, help="Render at this frame rate.", ), + option( + "--video-codec", + "video_codec", + default=None, + help="Video encoder used for cached segments (default: auto).", + ), + option( + "--pixel-format", + "pixel_format", + default=None, + help="Pixel format used for cached segments (default: auto).", + ), + option( + "--encoder-option", + "video_encoder_options", + multiple=True, + default=None, + callback=validate_encoder_options, + metavar="KEY=VALUE", + help="Set a codec option; repeat to set multiple options.", + ), option( "--max-inflight-encoders", type=IntRange(min=1), @@ -189,21 +255,7 @@ def validate_resolution( case_sensitive=False, ), help="Select a renderer for your Scene.", - default="cairo", - ), - option( - "-g", - "--save_pngs", - is_flag=True, - default=None, - help="Save each frame as png (Deprecated).", - ), - option( - "-i", - "--save_as_gif", default=None, - is_flag=True, - help="Save as a gif (Deprecated).", ), option( "--save_sections", @@ -215,6 +267,7 @@ def validate_resolution( "-t", "--transparent", is_flag=True, + default=None, help="Render scenes with alpha channel.", ), option( diff --git a/manim/manager.py b/manim/manager.py index dae957e53b..4a1acd4de3 100644 --- a/manim/manager.py +++ b/manim/manager.py @@ -7,12 +7,14 @@ import srt -from . import config, logger +from . import logger from .scene.section import DefaultSectionType from .utils.exceptions import EndSceneEarlyException, RerunSceneException from .utils.file_ops import open_media_file if TYPE_CHECKING: + from ._config.output import OutputSpec + from ._config.render_session import RenderSessionSpec from .animation.animation import Animation from .camera.camera import Camera from .mobject.mobject import Mobject, _AnimationBuilder @@ -85,6 +87,16 @@ def file_writer(self) -> SceneFileWriter: """Return the current renderer's file writer.""" return cast("SceneFileWriter", self.renderer.file_writer) + @property + def output_spec(self) -> OutputSpec: + """Return the immutable output intent captured for this session.""" + return self.session_spec.output + + @property + def session_spec(self) -> RenderSessionSpec: + """Return the immutable artifact, presentation, and execution intent.""" + return self.scene.session_spec + @property def time(self) -> float: """Return the current renderer time.""" @@ -133,6 +145,11 @@ def render(self, preview: bool = False) -> bool: ``False``. This matches the return value of :meth:`~manim.scene.scene.Scene.render`. """ + presentation = self.session_spec.presentation + open_after_render = preview or presentation.open_after_render + if open_after_render and not self.output_spec.enabled: + raise ValueError("Previewing after render requires a media artifact.") + self.setup() try: self.construct() @@ -158,12 +175,12 @@ def render(self, preview: bool = False) -> bool: self.tear_down() self.post_construct() - # If preview open up the render after rendering. - if preview: - config["preview"] = True - - if config["preview"] or config["show_in_file_browser"]: - open_media_file(self.file_writer) + if open_after_render or presentation.show_in_file_browser: + open_media_file( + self.file_writer, + preview=open_after_render, + show_in_file_browser=presentation.show_in_file_browser, + ) return False @@ -178,15 +195,34 @@ def construct(self) -> None: def post_construct(self) -> None: """Finalize output after scene construction and tear-down. - This asks the renderer to finish the scene and logs the number of played - animations. It intentionally runs after :meth:`tear_down` to preserve the - established render lifecycle. + This validates empty video output, asks the renderer to finish the scene, + and logs the number of played animations. It intentionally runs after + :meth:`tear_down` to preserve the established render lifecycle. """ + output = self.output_spec + empty_video_output = self.num_plays == 0 and output.is_video + if empty_video_output and not output.fallback_to_still: + raise RuntimeError( + f"{self.scene} has no play calls, so the explicitly requested " + f"{output.format.value.upper()} output cannot be produced. " + "Use --format=png to save its last frame.", + ) + # We have to reset these settings in case of multiple renders. self.renderer.scene_finished(self.scene) + if ( + empty_video_output + and output.fallback_to_still + and getattr(self.file_writer, "final_file_path", None) is not None + ): + logger.warning( + f"{self.scene} has no play calls. Automatic video output has " + "been saved as a PNG instead.", + ) + # Show info only if animations are rendered or to get image. - if self.num_plays or config["format"] == "png" or config["save_last_frame"]: + if self.num_plays or output.enabled: logger.info( f"Rendered {str(self.scene)}\nPlayed {self.num_plays} animations", ) diff --git a/manim/renderer/cairo_renderer.py b/manim/renderer/cairo_renderer.py index da7cab133d..755ecb0688 100644 --- a/manim/renderer/cairo_renderer.py +++ b/manim/renderer/cairo_renderer.py @@ -6,15 +6,19 @@ from manim.utils.hashing import get_hash_from_play_call from .. import config, logger +from .._config.video_encoder import video_encoder_fingerprint from ..camera.camera import Camera from ..mobject.mobject import Mobject, _AnimationBuilder from ..scene.scene_file_writer import SceneFileWriter from ..utils.exceptions import EndSceneEarlyException from ..utils.iterables import list_update +from .protocol import RendererCapabilities if TYPE_CHECKING: + from manim._config.render_session import RenderSessionSpec from manim.animation.animation import Animation from manim.scene.scene import Scene + from manim.scene.scene_file_writer import _SceneFileWriterSettings from ..typing import PixelArray @@ -33,6 +37,8 @@ class CairoRenderer: Time elapsed since initialisation of scene. """ + capabilities = RendererCapabilities(live_preview=False) + def __init__( self, file_writer_class: type[SceneFileWriter] = SceneFileWriter, @@ -53,11 +59,13 @@ def __init__( self.time = 0.0 self.static_image: PixelArray | None = None - def init_scene(self, scene: Scene) -> None: - self.file_writer: Any = self._file_writer_class( - self, - scene.__class__.__name__, - ) + def init_scene( + self, + scene: Scene, + session_spec: RenderSessionSpec, + file_writer_settings: _SceneFileWriterSettings, + ) -> None: + self.file_writer: Any = self._file_writer_class(file_writer_settings) def play( self, @@ -87,6 +95,11 @@ def play( self.camera, scene.animations, scene.mobjects, + backend="cairo", + encoder_fingerprint=video_encoder_fingerprint( + scene.session_spec.video_encoder, + ), + renderer_state=(), ) if self.file_writer.is_already_cached(hash_current_animation): logger.info( @@ -103,7 +116,10 @@ def play( {"h": str(self.animations_hashes[:5])}, ) - self.file_writer.begin_animation(not self.skip_animations) + self.file_writer.begin_animation( + not self.skip_animations, + animation_index=self.num_plays, + ) scene.begin_animations() # Save a static image, to avoid rendering non moving objects. @@ -192,7 +208,7 @@ def add_frame(self, frame: PixelArray, num_frames: int = 1) -> None: if self.skip_animations: return self.time += num_frames * dt - self.file_writer.write_frame(frame, num_frames=num_frames) + self.file_writer.write_frame(frame, repeat=num_frames) def freeze_current_frame(self, duration: float) -> None: """Adds a static frame to the movie for a given duration. The static frame is the current frame. @@ -252,7 +268,7 @@ def update_skipping_status(self) -> None: # there is always at least one section -> no out of bounds here if self.file_writer.sections[-1].skip_animations: self.skip_animations = True - if config["save_last_frame"]: + if self.file_writer.output_spec.is_still: self.skip_animations = True if ( config.from_animation_number > 0 @@ -267,17 +283,17 @@ def update_skipping_status(self) -> None: raise EndSceneEarlyException() def scene_finished(self, scene: Scene) -> None: - # If no animations in scene, render an image instead - if self.num_plays: + output = self.file_writer.output_spec + if self.num_plays and (output.is_video or output.is_image_sequence): self.file_writer.finish() - elif config.write_to_movie: - config.save_last_frame = True - config.write_to_movie = False - else: + elif not self.num_plays: self.static_image = None self.update_frame(scene) - if config["save_last_frame"]: - self.static_image = None - self.update_frame(scene) - self.file_writer.save_image(self.camera.get_image()) + # Automatically selected video output falls back to a last-frame PNG + # when a scene has no play calls. + if output.is_still or (not self.num_plays and output.fallback_to_still): + if self.num_plays: + self.static_image = None + self.update_frame(scene) + self.file_writer.save_image(self.get_frame()) diff --git a/manim/renderer/opengl_renderer.py b/manim/renderer/opengl_renderer.py index 80fc251f06..cbaee2bc9b 100644 --- a/manim/renderer/opengl_renderer.py +++ b/manim/renderer/opengl_renderer.py @@ -13,7 +13,7 @@ from PIL import Image from typing_extensions import override -from manim import config, logger +from manim import config from manim.mobject.opengl.opengl_mobject import ( OpenGLMobject, OpenGLPoint, @@ -36,6 +36,7 @@ rotation_matrix_transpose, rotation_matrix_transpose_from_quaternion, ) +from .protocol import RendererCapabilities from .shader import Mesh, Shader from .vectorized_mobject_rendering import ( render_opengl_vectorized_mobject_fill, @@ -46,9 +47,11 @@ from collections.abc import Iterable from typing import Self + from manim._config.render_session import RenderSessionSpec from manim.animation.animation import Animation from manim.mobject.mobject import Mobject, _AnimationBuilder from manim.scene.scene import Scene + from manim.scene.scene_file_writer import _SceneFileWriterSettings from manim.typing import ( FloatRGBA, PathFuncType, @@ -484,6 +487,8 @@ class OpenGLRenderer: The window used for previewing, if any. """ + capabilities = RendererCapabilities(live_preview=True) + def __init__( self, file_writer_class: type[SceneFileWriter] = SceneFileWriter, @@ -516,7 +521,12 @@ def __init__( self.path_to_texture_id: dict[str, int] = {} self.background_color = config["background_color"] - def init_scene(self, scene: Scene) -> None: + def init_scene( + self, + scene: Scene, + session_spec: RenderSessionSpec, + file_writer_settings: _SceneFileWriterSettings, + ) -> None: """ Initializes the OpenGL rendering context and related resources for the given scene. @@ -534,13 +544,12 @@ def init_scene(self, scene: Scene) -> None: """ self.partial_movie_files: list[str | None] = [] self.file_writer: SceneFileWriter = self._file_writer_class( - self, - scene.__class__.__name__, + file_writer_settings, ) self.scene = scene self.background_color = config["background_color"] - if self.should_create_window(): + if self.should_create_window(session_spec): from .opengl_renderer_window import Window self.window = Window(self) @@ -566,28 +575,13 @@ def init_scene(self, scene: Scene) -> None: moderngl.ONE, ) - def should_create_window(self) -> bool: + def should_create_window(self, session_spec: RenderSessionSpec) -> bool: """ Determine whether a window should be created for rendering based on the current configuration. - Notes - ----- - A windows is always created if the 'force_window' configuration is enabled. """ - if config["force_window"]: - logger.warning( - "'--force_window' is enabled, this is intended for debugging purposes " - "and may impact performance if used when outputting files", - ) - return True - return ( - config["preview"] - and not config["save_last_frame"] - and not config["format"] - and not config["write_to_movie"] - and not config["dry_run"] - ) + return session_spec.presentation.live_preview def get_pixel_shape(self) -> tuple[int, int] | None: """ @@ -802,6 +796,8 @@ def update_skipping_status(self) -> None: # there is always at least one section -> no out of bounds here if self.file_writer.sections[-1].skip_animations: self.skip_animations = True + if self.file_writer.output_spec.is_still: + self.skip_animations = True if ( config.from_animation_number > 0 and self.num_plays < config.from_animation_number @@ -839,16 +835,23 @@ def play( """ # TODO: Handle data locking / unlocking. self.animation_start_time = time.time() - self.file_writer.begin_animation(not self.skip_animations) + self.file_writer.begin_animation( + not self.skip_animations, + animation_index=self.num_plays, + ) scene.compile_animation_data(*animations, **kwargs) scene.begin_animations() if scene.is_current_animation_frozen_frame(): self.update_frame(scene) - if not self.skip_animations: + output = self.file_writer.output_spec + if not self.skip_animations and ( + output.is_video or output.is_image_sequence + ): self.file_writer.write_frame( - self, num_frames=int(config.frame_rate * scene.duration) + self.get_frame(), + repeat=int(config.frame_rate * scene.duration), ) if self.window is not None: @@ -908,7 +911,9 @@ def render( if self.skip_animations: return - self.file_writer.write_frame(self) + output = self.file_writer.output_spec + if output.is_video or output.is_image_sequence: + self.file_writer.write_frame(self.get_frame()) if self.window is not None: self.window.swap_buffers() @@ -956,46 +961,40 @@ def update_frame(self, scene: Scene) -> None: self.animation_elapsed_time = time.time() - self.animation_start_time def scene_finished(self, scene: Scene) -> None: - """ - Handle the finalization process after a scene has finished rendering. - - Performs the following actions: - - If any plays (animations) have occurred, finalizes the file writing process. - - If no plays have occurred but movie writing is enabled, disables - movie writing to avoid creating an empty movie file. - - If the configuration requires saving the last frame, - updates and saves the final image of the scene. + """Finalize configured output for the scene. Parameters ---------- - scene : Scene + scene The scene that has finished rendering. """ - # When num_plays is 0, no images have been output, so output a single - # image in this case - if self.num_plays > 0: + output = self.file_writer.output_spec + if self.num_plays > 0 and (output.is_video or output.is_image_sequence): self.file_writer.finish() - elif self.num_plays == 0 and config.write_to_movie: - config.write_to_movie = False + elif self.num_plays == 0: + # Keep the framebuffer useful for direct renderer access and + # graphical tests even when no media artifact was requested. + self.update_frame(scene) if self.should_save_last_frame(): - config.save_last_frame = True - self.update_frame(scene) - self.file_writer.save_image(self.get_image()) + if self.num_plays > 0: + self.update_frame(scene) + self.file_writer.save_image(self.get_frame()) def should_save_last_frame(self) -> bool: """ - Determine whether the last frame of the scene should be saved, - i.e. if one of the following conditions is met: - - The configuration option 'save_last_frame' is enabled. - - The scene is not in interactive mode. - - This is the first play (i.e., num_plays == 0). + Determine whether the last frame of the scene should be saved. + + This is true for explicit last-frame PNG output and for automatic video + output when the scene has no play calls. Interactive scenes do not use + the automatic fallback. """ - if config["save_last_frame"]: + output = self.file_writer.output_spec + if output.is_still: return True if self.scene.interactive_mode: return False - return self.num_plays == 0 + return self.num_plays == 0 and output.fallback_to_still def get_image(self) -> Image.Image: """ @@ -1140,8 +1139,7 @@ def get_frame(self) -> RGBAPixelArray: result_dimensions = (pixel_shape[1], pixel_shape[0], 4) np_buf = np.frombuffer(raw, dtype="uint8").reshape(result_dimensions) - np_buf = np.flipud(np_buf) - return np_buf + return np.flipud(np_buf).copy() # Returns offset from the bottom left corner in pixels. # top_left flag should be set to True when using a GUI framework diff --git a/manim/renderer/protocol.py b/manim/renderer/protocol.py new file mode 100644 index 0000000000..87a0d7ebf7 --- /dev/null +++ b/manim/renderer/protocol.py @@ -0,0 +1,14 @@ +"""Shared renderer feature declarations.""" + +from __future__ import annotations + +__all__ = ["RendererCapabilities"] + +from dataclasses import dataclass + + +@dataclass(frozen=True, slots=True) +class RendererCapabilities: + """Optional session features implemented by a renderer.""" + + live_preview: bool = False diff --git a/manim/scene/scene.py b/manim/scene/scene.py index b4ef54f38f..29f69d5c52 100644 --- a/manim/scene/scene.py +++ b/manim/scene/scene.py @@ -43,6 +43,15 @@ from manim.mobject.opengl.opengl_mobject import OpenGLPoint from .. import config, logger +from .._config.logger_utils import set_file_logger +from .._config.output_plan import ( + resolve_file_log_path, + resolve_media_layout, + resolve_module_name, + resolve_output_plan, + resolve_requested_output_name, +) +from .._config.render_session import resolve_render_session from ..animation.animation import Animation, Wait, prepare_animation from ..camera.camera import Camera from ..constants import * @@ -50,6 +59,7 @@ from ..renderer.cairo_renderer import CairoRenderer from ..renderer.opengl_renderer import OpenGLCamera, OpenGLMobject, OpenGLRenderer from ..renderer.shader import Object3D +from ..scene.scene_file_writer import _SceneFileWriterSettings from ..utils import opengl, space_ops from ..utils.exceptions import RerunSceneException from ..utils.family import extract_mobject_family_members @@ -213,7 +223,53 @@ def __init__( ) else: self.renderer = renderer - self.renderer.init_scene(self) + self.session_spec = resolve_render_session( + config, + self.renderer.capabilities, + renderer_name=type(self.renderer).__name__, + ) + scene_name = type(self).__name__ + module_name = resolve_module_name(config) + working_directory = Path.cwd() + media_layout = resolve_media_layout( + config, + self.session_spec.output, + module_name=module_name, + scene_name=scene_name, + working_directory=working_directory, + ) + self.output_plan = resolve_output_plan( + media_layout, + self.session_spec.output, + scene_name=scene_name, + requested_output_name=resolve_requested_output_name(config), + ) + assets_dir = config.get_dir("assets_dir") + if assets_dir is None: + assets_dir = working_directory + elif not assets_dir.is_absolute(): + assets_dir = working_directory / assets_dir + self.file_writer_settings = _SceneFileWriterSettings( + plan=self.output_plan, + video_encoder=self.session_spec.video_encoder, + max_inflight_encoders=config.max_inflight_encoders, + encoder_queue_size=config.encoder_queue_size, + max_files_cached=config.max_files_cached, + assets_dir=assets_dir.absolute(), + ) + self._log_file_path = resolve_file_log_path( + media_layout, + module_name=module_name, + scene_name=scene_name, + ) + if self._log_file_path is not None: + self._log_file_path.parent.mkdir(parents=True, exist_ok=True) + set_file_logger(self._log_file_path) + self.renderer.init_scene( + self, + self.session_spec, + self.file_writer_settings, + ) self.mobjects: list[Mobject] = [] # TODO, remove need for foreground mobjects @@ -1362,20 +1418,14 @@ def play_internal(self, skip_rendering: bool = False) -> None: def check_interactive_embed_is_valid(self) -> bool: assert isinstance(self.renderer, OpenGLRenderer) - if config["force_window"]: - return True if self.skip_animation_preview: logger.warning( "Disabling interactive embed as 'skip_animation_preview' is enabled", ) return False - elif config["write_to_movie"]: - logger.warning("Disabling interactive embed as 'write_to_movie' is enabled") - return False - elif config["format"]: + elif self.renderer.file_writer.output_spec.enabled: logger.warning( - "Disabling interactive embed as '--format' is set as " - + config["format"], + "Disabling interactive embed while media output is enabled", ) return False elif not self.renderer.window: @@ -1551,10 +1601,10 @@ def interact(self, shell: Any, keyboard_thread: threading.Thread) -> None: def embed(self) -> None: assert isinstance(self.renderer, OpenGLRenderer) - if not config["preview"]: - logger.warning("Called embed() while no preview window is available.") + if not self.session_spec.presentation.live_preview: + logger.warning("Called embed() while no live preview window is available.") return - if config["write_to_movie"]: + if self.renderer.file_writer.output_spec.enabled: logger.warning("embed() is skipped while writing to a file.") return diff --git a/manim/scene/scene_file_writer.py b/manim/scene/scene_file_writer.py index 17661c23aa..9617b57ef0 100644 --- a/manim/scene/scene_file_writer.py +++ b/manim/scene/scene_file_writer.py @@ -1,4 +1,4 @@ -"""The interface between scenes and ffmpeg.""" +"""Scene output coordination and media-artifact assembly.""" from __future__ import annotations @@ -8,7 +8,8 @@ import shutil import warnings from contextlib import suppress -from fractions import Fraction +from dataclasses import dataclass +from io import BytesIO from pathlib import Path from queue import Queue from tempfile import NamedTemporaryFile, _TemporaryFileWrapper @@ -33,45 +34,17 @@ from manim import __version__ -from .. import config, logger -from .._config.logger_utils import set_file_logger -from ..constants import RendererType -from ..utils.file_ops import ( - add_extension_if_not_present, - add_version_before_extension, - guarantee_existence, - is_gif_format, - is_png_format, - modify_atime, - write_to_movie, -) +from .. import logger +from .._config.output_plan import OutputPlan +from .._config.video_encoder import VideoEncoderSpec +from ..utils.caching import prune_segment_cache +from ..utils.file_ops import modify_atime from ..utils.sounds import get_full_sound_file_path from .section import DefaultSectionType, Section +from .video_segment_encoder import VideoSegmentEncoder if TYPE_CHECKING: - from av.container.output import OutputContainer - from av.stream import Stream - - from manim.renderer.cairo_renderer import CairoRenderer - from manim.renderer.opengl_renderer import OpenGLRenderer - from manim.typing import PixelArray, StrPath - - -def to_av_frame_rate(fps: float) -> Fraction: - epsilon1 = 1e-4 - epsilon2 = 0.02 - - if isinstance(fps, int): - (num, denom) = (fps, 1) - elif abs(fps - round(fps)) < epsilon1: - (num, denom) = (round(fps), 1) - else: - denom = 1001 - num = round(fps * denom / 1000) * 1000 - if abs(fps - num / denom) >= epsilon2: - raise ValueError("invalid frame rate") - - return Fraction(num, denom) + from manim.typing import RGBAPixelArray, StrPath def convert_audio( @@ -92,28 +65,28 @@ def convert_audio( class _PartialMovieEncodeJob: - """Encode and write the frames for one partial movie file.""" + """Run one segment encoder on a dedicated worker thread.""" def __init__( self, - path: StrPath, + *, animation_index: int, - container: OutputContainer, - stream: Stream, + encoder: VideoSegmentEncoder, frame_queue_size: int, ) -> None: - self.path = path + self.path = encoder.target self.animation_index = animation_index - self.container = container - self.stream = stream + self.encoder = encoder # A size of 0 preserves the unbounded queue used by serial encoding. # Parallel encoding uses a bounded queue; at the default capacity, eight # 1080p RGBA frames occupy about 66 MB per job. The worker drains through # the sentinel after an exception, so a bounded queue cannot deadlock. - self.queue: Queue[tuple[int, PixelArray | None]] = Queue( + self.queue: Queue[tuple[int, RGBAPixelArray | None]] = Queue( maxsize=frame_queue_size, ) self._exception: BaseException | None = None + self._sealed = False + self._abort_requested = False self.thread = Thread( target=self._listen_and_write, name=f"partial-movie-encoder-{animation_index}", @@ -129,113 +102,143 @@ def failed(self) -> bool: """Whether the worker has captured an exception.""" return self._exception is not None + def _abort_encoder(self) -> None: + try: + self.encoder.abort() + except BaseException as exception: + logger.warning( + "Failed to clean up incomplete segment %(path)s: %(error)s", + {"path": f"'{self.path}'", "error": exception}, + ) + self._capture_exception(exception) + def _listen_and_write(self) -> None: while True: - num_frames, frame_data = self.queue.get() + repeat, frame_data = self.queue.get() if frame_data is None: break if self._exception is not None: continue try: - self._encode_and_write_frame(frame_data, num_frames) + self.encoder.write_frame(frame_data, repeat=repeat) except BaseException as exception: self._capture_exception(exception) + if self._abort_requested or self._exception is not None: + self._abort_encoder() + return + try: - for packet in self.stream.encode(): - self.container.mux(packet) + self.encoder.finish() except BaseException as exception: self._capture_exception(exception) - finally: - try: - self.container.close() - except BaseException as exception: - self._capture_exception(exception) + self._abort_encoder() - def _encode_and_write_frame(self, frame: PixelArray, num_frames: int) -> None: - for _ in range(num_frames): - # Notes: precomputing reusing packets does not work! - # I.e., you cannot do `packets = encode(...)` - # and reuse it, as it seems that `mux(...)` - # consumes the packet. - # The same issue applies for `av_frame`, - # reusing it renders weird-looking frames. - av_frame = av.VideoFrame.from_ndarray(frame, format="rgba") - for packet in self.stream.encode(av_frame): - self.container.mux(packet) - - def put(self, num_frames: int, frame: PixelArray) -> None: + def put(self, repeat: int, frame: RGBAPixelArray) -> None: """Add a frame to the encoding queue.""" - self.queue.put((num_frames, frame)) + self.queue.put((repeat, frame)) def seal(self) -> None: """Signal that no more frames will be added.""" - self.queue.put((-1, None)) + if not self._sealed: + self._sealed = True + self.queue.put((-1, None)) + + def abort(self) -> None: + """Signal that the segment must be discarded.""" + self._abort_requested = True + self.seal() def join(self) -> None: """Wait for encoding to finish and propagate worker failures.""" self.thread.join() if self._exception is not None: - # A failed encode may leave a structurally valid but truncated - # file behind; remove it so a later run cannot cache-hit it. - try: - Path(self.path).unlink(missing_ok=True) - except OSError as cleanup_error: - logger.warning( - "Failed to remove incomplete partial movie file %(path)s: " - "%(error)s", - {"path": f"'{self.path}'", "error": cleanup_error}, - ) raise self._exception + if not self._abort_requested: + logger.info( + f"Animation {self.animation_index} : Partial movie file written in %(path)s", + {"path": f"'{self.path}'"}, + ) - logger.info( - f"Animation {self.animation_index} : Partial movie file written in %(path)s", - {"path": f"'{self.path}'"}, - ) +@dataclass(frozen=True, slots=True) +class _SceneFileWriterSettings: + """Immutable inputs consumed by one :class:`SceneFileWriter`. -class SceneFileWriter: - """SceneFileWriter is the object that actually writes the animations - played, into video files, using FFMPEG. - This is mostly for Manim's internal use. You will rarely, if ever, - have to use the methods for this class, unless tinkering with the very - fabric of Manim's reality. + The settings contain resolved output paths and segment encoding, bounded + encoder-pool limits, cache maintenance, and the sound-asset search root. + """ - Attributes - ---------- - sections : list of :class:`.Section` - used to segment scene + plan: OutputPlan + video_encoder: VideoEncoderSpec | None + max_inflight_encoders: int + encoder_queue_size: int + max_files_cached: int + assets_dir: Path + + def __post_init__(self) -> None: + output = self.plan.output + if output.is_video != (self.video_encoder is not None): + raise ValueError( + "Video output and resolved video encoder settings must be provided together.", + ) + expected_segment_extension = ( + output.segment_extension if output.is_video else None + ) + if self.plan.segment_extension != expected_segment_extension: + raise ValueError( + "The output plan segment extension does not match its output specification.", + ) + if ( + self.video_encoder is not None + and f".{self.video_encoder.container_format}" != expected_segment_extension + ): + raise ValueError( + "The video encoder container does not match the output plan.", + ) + if self.max_inflight_encoders <= 0: + raise ValueError("max_inflight_encoders must be positive.") + if self.encoder_queue_size <= 0: + raise ValueError("encoder_queue_size must be positive.") + if self.max_files_cached < -1: + raise ValueError("max_files_cached must be non-negative or -1.") + if not self.assets_dir.is_absolute(): + raise ValueError("assets_dir must be absolute.") - sections_output_dir : :class:`pathlib.Path` - where are section videos stored - output_name : str - name of movie without extension and basis for section video names +class SceneFileWriter: + """Coordinate segment jobs and assemble one scene's media artifacts. - Some useful attributes are: - "write_to_movie" (bool=False) - Whether or not to write the animations into a video file. - "movie_file_extension" (str=".mp4") - The file-type extension of the outputted video. - "partial_movie_files" - List of all the partial-movie files. + The writer receives immutable resolved settings and concrete top-left-origin + RGBA arrays. Ownership of each array passed to + :meth:`write_frame` transfers to the writer; callers must not mutate or reuse + it afterward. For video output the writer coordinates queued + :class:`.VideoSegmentEncoder` jobs, then assembles their silent cached + segments with optional audio, sections, and subcaptions. It also writes + still images and PNG sequences described by the output plan. - """ + Parameters + ---------- + settings + Resolved output, encoding, pool, cache, and asset-search settings. - force_output_as_scene_name = False + Attributes + ---------- + sections + Ordered section metadata for the scene. + partial_movie_files + Segment paths in animation order, including ``None`` for skipped plays. + """ - def __init__( - self, - renderer: CairoRenderer | OpenGLRenderer, - scene_name: str, - **kwargs: Any, - ) -> None: - self.renderer = renderer + def __init__(self, settings: _SceneFileWriterSettings) -> None: + self.settings = settings + self.output_spec = settings.plan.output + self.output_plan = settings.plan + self.video_encoder = settings.video_encoder self._inflight_encode_jobs: list[_PartialMovieEncodeJob] = [] self._inflight_by_path: dict[str, _PartialMovieEncodeJob] = {} self._current_encode_job: _PartialMovieEncodeJob | None = None - self.init_output_directories(scene_name) self.init_audio() self.frame_count = 0 self.partial_movie_files: list[str | None] = [] @@ -247,82 +250,59 @@ def __init__( name="autocreated", type_=DefaultSectionType.NORMAL, skip_animations=False ) - def init_output_directories(self, scene_name: str) -> None: - """Initialise output directories. - - Notes - ----- - The directories are read from ``config``, for example - ``config['media_dir']``. If the target directories don't already - exist, they will be created. - - """ - if config["dry_run"]: # in dry-run mode there is no output - return - - module_name = config.get_dir("input_file").stem if config["input_file"] else "" - - if SceneFileWriter.force_output_as_scene_name: - self.output_name = Path(scene_name) - elif config["output_file"] and not config["write_all"]: - self.output_name = config.get_dir("output_file") - else: - self.output_name = Path(scene_name) - - if config["media_dir"]: - image_dir = guarantee_existence( - config.get_dir( - "images_dir", module_name=module_name, scene_name=scene_name - ), - ) - self.image_file_path = image_dir / add_extension_if_not_present( - self.output_name, ".png" - ) - - if write_to_movie(): - movie_dir = guarantee_existence( - config.get_dir( - "video_dir", module_name=module_name, scene_name=scene_name - ), - ) - self.movie_file_path = movie_dir / add_extension_if_not_present( - self.output_name, config["movie_file_extension"] - ) + @property + def output_name(self) -> Path: + """Return the planned logical output stem.""" + return Path(self.output_plan.output_stem) - # TODO: /dev/null would be good in case sections_output_dir is used without being set (doesn't work on Windows), everyone likes defensive programming, right? - self.sections_output_dir = Path("") - if config.save_sections: - self.sections_output_dir = guarantee_existence( - config.get_dir( - "sections_dir", module_name=module_name, scene_name=scene_name - ) - ) + @property + def image_file_path(self) -> Path: + """Return the planned still or video-fallback image path.""" + if self.output_spec.is_image_sequence: + return self.image_sequence_directory.with_suffix(".png") + path = ( + self.output_plan.primary_artifact + if self.output_spec.is_still + else self.output_plan.fallback_image + ) + if path is None: + raise AttributeError("This output plan does not contain an image path.") + return path - if is_gif_format(): - self.gif_file_path = add_extension_if_not_present( - self.output_name, ".gif" - ) + @property + def image_sequence_directory(self) -> Path: + """Return the planned PNG-sequence directory.""" + path = self.output_plan.image_sequence_dir + if path is None: + raise AttributeError("This output plan does not contain an image sequence.") + return path - if not config["output_file"]: - self.gif_file_path = add_version_before_extension( - self.gif_file_path - ) + @property + def movie_file_path(self) -> Path: + """Return the planned primary video artifact path.""" + if not self.output_spec.is_video or self.output_plan.primary_artifact is None: + raise AttributeError("This output plan does not contain a video artifact.") + return self.output_plan.primary_artifact - self.gif_file_path = movie_dir / self.gif_file_path + @property + def gif_file_path(self) -> Path: + """Return the planned GIF artifact path.""" + if not self.output_spec.is_gif: + raise AttributeError("This output plan does not contain a GIF artifact.") + return self.movie_file_path - self.partial_movie_directory = guarantee_existence( - config.get_dir( - "partial_movie_dir", - scene_name=scene_name, - module_name=module_name, - ), - ) + @property + def sections_output_dir(self) -> Path: + """Return the planned sections directory or an empty path.""" + return self.output_plan.sections_dir or Path("") - if config["log_to_file"]: - log_dir = guarantee_existence(config.get_dir("log_dir")) - set_file_logger( - scene_name=scene_name, module_name=module_name, log_dir=log_dir - ) + @property + def partial_movie_directory(self) -> Path: + """Return the planned silent-segment cache directory.""" + path = self.output_plan.segment_cache_dir + if path is None: + raise AttributeError("This output plan does not contain video segments.") + return path def finish_last_section(self) -> None: """Delete current section if it is empty.""" @@ -336,14 +316,13 @@ def next_section(self, name: str, type_: str, skip_animations: bool) -> None: # images don't support sections section_video: str | None = None # don't save when None - if ( - not config.dry_run - and write_to_movie() - and config.save_sections - and not skip_animations - ): - # relative to index file - section_video = f"{self.output_name}_{len(self.sections):04}_{name}{config.movie_file_extension}" + if self.output_spec.save_sections and not skip_animations: + section_path = self.output_plan.section_path(len(self.sections), name) + assert self.output_plan.sections_dir is not None + # Section stores paths relative to its index file. + section_video = section_path.relative_to( + self.output_plan.sections_dir, + ).as_posix() self.sections.append( Section( @@ -355,64 +334,28 @@ def next_section(self, name: str, type_: str, skip_animations: bool) -> None: ) def add_partial_movie_file(self, hash_animation: str | None) -> None: - """Adds a new partial movie file path to ``scene.partial_movie_files`` - and current section from a hash. + """Append a planned segment path to the writer and current section. - This method will compute the path from the hash. In addition to that it - adds the new animation to the current section. + The list retains one entry per animation so explicit animation indices + select the corresponding segment. Parameters ---------- hash_animation Hash of the animation. """ - if not hasattr(self, "partial_movie_directory") or not write_to_movie(): + if not self.output_spec.is_video: return - # None has to be added to partial_movie_files to keep the right index with scene.num_plays. - # i.e if an animation is skipped, scene.num_plays is still incremented and we add an element to partial_movie_file be even with num_plays. + # Skipped animations retain a placeholder to preserve index alignment. if hash_animation is None: self.partial_movie_files.append(None) self.sections[-1].partial_movie_files.append(None) else: - new_partial_movie_file = str( - self.partial_movie_directory - / f"{hash_animation}{config['movie_file_extension']}" - ) + new_partial_movie_file = str(self.output_plan.segment_path(hash_animation)) self.partial_movie_files.append(new_partial_movie_file) self.sections[-1].partial_movie_files.append(new_partial_movie_file) - def get_resolution_directory(self) -> str: - """Get the name of the resolution directory directly containing - the video file. - - This method gets the name of the directory that immediately contains the - video file. This name is ``p``. - For example, if you are rendering an 854x480 px animation at 15fps, - the name of the directory that immediately contains the video, file - will be ``480p15``. - - The file structure should look something like:: - - MEDIA_DIR - |--Tex - |--texts - |--videos - |-- - |--p - |--partial_movie_files - |--.mp4 - |--.srt - - Returns - ------- - :class:`str` - The name of the directory. - """ - pixel_height = config["pixel_height"] - frame_rate = config["frame_rate"] - return f"{pixel_height}p{frame_rate}" - # Sound def init_audio(self) -> None: """Preps the writer for adding audio to the movie.""" @@ -490,7 +433,7 @@ def add_sound( used there can be referenced here. """ - file_path = get_full_sound_file_path(sound_file) + file_path = get_full_sound_file_path(sound_file, self.settings.assets_dir) # we assume files with .wav / .raw suffix are actually # .wav and .raw files, respectively. if file_path.suffix not in (".wav", ".raw"): @@ -510,184 +453,139 @@ def add_sound( # Writers def begin_animation( - self, allow_write: bool = False, file_path: StrPath | None = None + self, + allow_write: bool = False, + *, + animation_index: int, + file_path: StrPath | None = None, ) -> None: - """Used internally by manim to stream the animation to FFMPEG for - displaying or writing to a file. + """Start a segment job for one animation when video writing is enabled. Parameters ---------- allow_write - Whether or not to write to a video file. + Whether this animation needs a new segment. + animation_index + Scene-local animation index used to select and label the segment. + file_path + Explicit segment target, or ``None`` to use the planned cache path. """ - if write_to_movie() and allow_write: - self.open_partial_movie_stream(file_path=file_path) + if self.output_spec.is_video and allow_write: + self.open_partial_movie_stream( + animation_index=animation_index, + file_path=file_path, + ) def end_animation(self, allow_write: bool = False) -> None: - """Internally used by Manim to stop streaming to FFMPEG gracefully. + """Seal the current segment job when video writing is enabled. Parameters ---------- allow_write - Whether or not to write to a video file. + Whether the current animation has an open segment job. """ - if write_to_movie() and allow_write: + if self.output_spec.is_video and allow_write: self.close_partial_movie_stream() def write_frame( - self, frame_or_renderer: PixelArray | OpenGLRenderer, num_frames: int = 1 + self, + pixels: RGBAPixelArray, + *, + repeat: int = 1, ) -> None: - """Used internally by Manim to write a frame to the FFMPEG input buffer. + """Take ownership of one top-left C-contiguous ``uint8`` RGBA frame. - Parameters - ---------- - frame_or_renderer - Pixel array of the frame. - num_frames - The number of times to write frame. + The caller must not mutate or reuse ``pixels`` after this method returns + because video encoding can consume the array asynchronously. """ - if write_to_movie(): - if isinstance(frame_or_renderer, np.ndarray): - frame = frame_or_renderer - else: - frame = ( - frame_or_renderer.get_frame() - if config.renderer == RendererType.OPENGL - else frame_or_renderer - ) - + if self.output_spec.is_video: job = self._current_encode_job if job is None: - # Interactive OpenGL rendering emits frames outside an open - # partial movie stream; drop them silently. + # Presentation rendering can emit frames outside an open + # segment; such frames do not belong to file output. return if job.failed: # Surface the failure at the first write after it was captured; - # join() unlinks the partial and re-raises. + # the worker discards the partial before join() re-raises. job.seal() self._current_encode_job = None job.join() - job.put(num_frames, frame) + job.put(repeat, pixels) - if is_png_format() and not config["dry_run"]: - if isinstance(frame_or_renderer, np.ndarray): - image = Image.fromarray(frame_or_renderer) - else: - image = ( - frame_or_renderer.get_image() - if config.renderer == RendererType.OPENGL - else Image.fromarray(frame_or_renderer) - ) - target_dir = self.image_file_path.parent / self.image_file_path.stem - extension = self.image_file_path.suffix - self.output_image( - image, - target_dir, - extension, - config["zero_pad"], - ) + if self.output_spec.is_image_sequence: + self.output_image(Image.fromarray(pixels)) - def output_image( - self, image: Image.Image, target_dir: StrPath, ext: str, zero_pad: int - ) -> None: - if zero_pad: - image.save(f"{target_dir}{str(self.frame_count).zfill(zero_pad)}{ext}") - else: - image.save(f"{target_dir}{self.frame_count}{ext}") + def output_image(self, image: Image.Image) -> None: + file_path = self.output_plan.image_frame_path(self.frame_count) + file_path.parent.mkdir(parents=True, exist_ok=True) + image.save(file_path) self.frame_count += 1 - def save_image(self, image: Image.Image) -> None: - """This method saves the image passed to it in the default image directory. - - Parameters - ---------- - image - The pixel array of the image to save. - """ - if config["dry_run"]: + def save_image(self, pixels: RGBAPixelArray) -> None: + """Save one RGBA frame to the planned still-image path.""" + if not self.output_spec.enabled: return - if not config["output_file"]: - self.image_file_path = add_version_before_extension(self.image_file_path) - - image.save(self.image_file_path) + self.image_file_path.parent.mkdir(parents=True, exist_ok=True) + Image.fromarray(pixels).save(self.image_file_path) self.print_file_ready_message(self.image_file_path) def finish(self) -> None: - """Finishes writing to the FFMPEG buffer or writing images to output directory. - Combines the partial movie files into the whole scene. - If save_last_frame is True, saves the last frame in the default image directory. - """ - if write_to_movie(): + """Drain segment jobs and assemble the configured time-based output.""" + if self.output_spec.is_video: self.join_all_encode_jobs() self.combine_to_movie() - if config.save_sections: + if self.output_spec.save_sections: self.combine_to_section_videos() # Cache cleanup runs after the in-flight encode jobs have been drained. - if config["flush_cache"]: - self.flush_cache_directory() - else: - self.clean_cache() - elif is_png_format() and not config["dry_run"]: - target_dir = self.image_file_path.parent / self.image_file_path.stem + prune_segment_cache( + self.partial_movie_directory, + self.settings.max_files_cached, + ) + elif self.output_spec.is_image_sequence: + target_dir = self.image_sequence_directory + self.final_file_path = target_dir logger.info("\n%i images ready at %s\n", self.frame_count, str(target_dir)) if self.subcaptions: self.write_subcaption_file() - def open_partial_movie_stream(self, file_path: StrPath | None = None) -> None: - """Open a container holding a video stream. + def _create_segment_encoder(self, target: Path) -> VideoSegmentEncoder: + encoder = self.video_encoder + if encoder is None: + raise RuntimeError("Video segment encoding requires resolved settings.") + return VideoSegmentEncoder(target=target, spec=encoder) - This is used internally by Manim initialize the container holding - the video stream of a partial movie file. - """ + def open_partial_movie_stream( + self, + *, + animation_index: int, + file_path: StrPath | None = None, + ) -> None: + """Create a queued encoder job for one planned video segment.""" + if self._current_encode_job is not None: + raise RuntimeError( + "Cannot open a video segment while another segment is still open.", + ) if file_path is None: - file_path = self.partial_movie_files[self.renderer.num_plays] + file_path = self.partial_movie_files[animation_index] if file_path is None: raise RuntimeError( "open_partial_movie_stream() called for a play that has no " "partial movie file path.", ) + file_path = Path(file_path) + file_path.parent.mkdir(parents=True, exist_ok=True) path_key = str(file_path) if path_key in self._inflight_by_path: self._join_job_and_drain_on_failure(self._inflight_by_path[path_key]) - self.partial_movie_file_path = file_path - - fps = to_av_frame_rate(config.frame_rate) - - partial_movie_file_codec = "libx264" - partial_movie_file_pix_fmt = "yuv420p" - av_options = { - "an": "1", # ffmpeg: -an, no audio - "crf": "23", # ffmpeg: -crf, constant rate factor (improved bitrate) - } - - if config.movie_file_extension == ".webm": - partial_movie_file_codec = "libvpx-vp9" - av_options["-auto-alt-ref"] = "1" - if config.transparent: - partial_movie_file_pix_fmt = "yuva420p" - - elif config.transparent: - partial_movie_file_codec = "qtrle" - partial_movie_file_pix_fmt = "argb" - - video_container = av.open(file_path, mode="w") - stream = video_container.add_stream( - partial_movie_file_codec, - rate=fps, - options=av_options, - ) - stream.pix_fmt = partial_movie_file_pix_fmt - stream.width = config.pixel_width - stream.height = config.pixel_height - + segment_encoder = self._create_segment_encoder(file_path) frame_queue_size = ( - 0 if config.max_inflight_encoders == 1 else config.encoder_queue_size + 0 + if self.settings.max_inflight_encoders == 1 + else self.settings.encoder_queue_size ) self._current_encode_job = _PartialMovieEncodeJob( - path=file_path, - animation_index=self.renderer.num_plays, - container=video_container, - stream=stream, + animation_index=animation_index, + encoder=segment_encoder, frame_queue_size=frame_queue_size, ) @@ -727,24 +625,18 @@ def join_all_encode_jobs(self) -> None: raise first_exception def abort_encode_jobs(self, reraise_encoder_failures: bool = False) -> None: - """Tear down encode jobs after an aborted or rerun render. - - Seals the current job so its worker can exit (a non-daemon thread - blocked on the queue would hang the process at exit), then deletes its - partial file unconditionally: an aborted partial is structurally valid - but truncated, so leaving it behind produces an erroneous cache hit on - a later run. Sealed in-flight jobs are then drained. With - ``reraise_encoder_failures=False`` (a render exception is already - propagating) drain failures are logged, not raised; with ``True`` - (rerun path -- no primary exception exists) the first drain failure - propagates so corrupt completed partials cannot be silently ignored. + """Discard the current segment and drain completed encode jobs. + + When ``reraise_encoder_failures`` is true, the first encoder failure is + propagated. Otherwise failures are logged so an active render exception + remains primary. """ current_exception: BaseException | None = None job = self._current_encode_job if job is not None: - # Seal before clearing: an interrupt landing between the two - # statements must not orphan the worker. - job.seal() + # Request abort before clearing: an interrupt between these + # statements must not orphan a worker blocked on its queue. + job.abort() self._current_encode_job = None job.thread.join() current_exception = job._exception @@ -754,18 +646,11 @@ def abort_encode_jobs(self, reraise_encoder_failures: bool = False) -> None: job.animation_index, exc_info=current_exception, ) - try: - Path(job.path).unlink(missing_ok=True) + else: logger.info( "Discarded partial movie file of aborted animation %(index)d", {"index": job.animation_index}, ) - except OSError as cleanup_error: - logger.warning( - "Failed to remove incomplete partial movie file %(path)s: " - "%(error)s", - {"path": f"'{job.path}'", "error": cleanup_error}, - ) if reraise_encoder_failures: self.join_all_encode_jobs() if current_exception is not None: @@ -779,12 +664,7 @@ def abort_encode_jobs(self, reraise_encoder_failures: bool = False) -> None: logger.exception("Encoder failure while aborting render") def close_partial_movie_stream(self) -> None: - """Close the currently opened video container. - - Used internally by Manim to first flush the remaining packages - in the video stream holding a partial file, and then close - the corresponding container. - """ + """Seal the current segment and enforce the in-flight job limit.""" job = self._current_encode_job if job is None: raise RuntimeError( @@ -796,7 +676,7 @@ def close_partial_movie_stream(self) -> None: self._inflight_by_path[str(job.path)] = job self._current_encode_job = None - while len(self._inflight_encode_jobs) >= config.max_inflight_encoders: + while len(self._inflight_encode_jobs) >= self.settings.max_inflight_encoders: self._join_job_and_drain_on_failure(self._inflight_encode_jobs[0]) def is_already_cached(self, hash_invocation: str) -> bool: @@ -812,17 +692,49 @@ def is_already_cached(self, hash_invocation: str) -> bool: :class:`bool` Whether the file exists. """ - if not hasattr(self, "partial_movie_directory") or not write_to_movie(): + if not self.output_spec.is_video: return False - path = ( - self.partial_movie_directory - / f"{hash_invocation}{config['movie_file_extension']}" - ) + path = self.output_plan.segment_path(hash_invocation) path_key = str(path) if path_key in self._inflight_by_path: self._join_job_and_drain_on_failure(self._inflight_by_path[path_key]) return path.exists() + @staticmethod + def _concat_manifest_bytes(input_files: list[str]) -> bytes: + """Return a complete FFmpeg concat manifest for ``input_files``.""" + manifest_text = ( + "# This file records the segment order used by Manim.\n" + + "".join( + f"file 'file:{Path(file_path).as_posix()}'\n" + for file_path in input_files + ) + ) + return manifest_text.encode("utf-8") + + def _write_concat_manifest(self, input_files: list[str]) -> None: + """Atomically persist the complete scene segment order for diagnostics.""" + manifest_path = self.output_plan.concat_manifest + assert manifest_path is not None + manifest_path.parent.mkdir(parents=True, exist_ok=True) + temporary_path: Path | None = None + try: + with NamedTemporaryFile( + mode="wb", + dir=manifest_path.parent, + prefix=f".{manifest_path.name}.", + suffix=".tmp", + delete=False, + ) as temporary_file: + temporary_path = Path(temporary_file.name) + temporary_file.write(self._concat_manifest_bytes(input_files)) + temporary_path.replace(manifest_path) + except BaseException: + if temporary_path is not None: + with suppress(OSError): + temporary_path.unlink(missing_ok=True) + raise + def combine_files( self, input_files: list[str], @@ -830,16 +742,12 @@ def combine_files( create_gif: bool = False, includes_sound: bool = False, ) -> None: - file_list = self.partial_movie_directory / "partial_movie_file_list.txt" + output_file.parent.mkdir(parents=True, exist_ok=True) logger.debug( f"Partial movie files to combine ({len(input_files)} files): %(p)s", {"p": input_files[:5]}, ) - with file_list.open("w", encoding="utf-8") as fp: - fp.write("# This file is used internally by FFMPEG.\n") - for pf_path in input_files: - pf_path = Path(pf_path).as_posix() - fp.write(f"file 'file:{pf_path}'\n") + manifest = BytesIO(self._concat_manifest_bytes(input_files)) av_options = { "safe": "0", # needed to read files @@ -849,7 +757,9 @@ def combine_files( av_options["an"] = "1" partial_movies_input = av.open( - str(file_list), options=av_options, format="concat" + manifest, + options=av_options, + format="concat", ) partial_movies_stream = partial_movies_input.streams.video[0] output_container = av.open(str(output_file), mode="w") @@ -866,11 +776,13 @@ def combine_files( codec_name="gif", ) output_stream.pix_fmt = "rgb8" - if config.transparent: + if self.output_spec.transparent: output_stream.pix_fmt = "pal8" - output_stream.width = config.pixel_width - output_stream.height = config.pixel_height - output_stream.rate = to_av_frame_rate(config.frame_rate) + encoder = self.video_encoder + assert encoder is not None + output_stream.width = encoder.width + output_stream.height = encoder.height + output_stream.rate = encoder.frame_rate graph = av.filter.Graph() input_buffer = graph.add_buffer(template=partial_movies_stream) split = graph.add("split") @@ -912,7 +824,10 @@ def combine_files( output_stream = output_container.add_stream_from_template( template=partial_movies_stream, ) - if config.transparent and config.movie_file_extension == ".webm": + if ( + self.output_spec.transparent + and self.output_spec.segment_extension == ".webm" + ): output_stream.pix_fmt = "yuva420p" for packet in partial_movies_input.demux(partial_movies_stream): # We need to skip the "flushing" packets that `demux` generates. @@ -928,6 +843,7 @@ def combine_files( partial_movies_input.close() output_container.close() + manifest.close() def combine_to_movie(self) -> None: """Used internally by Manim to combine the separate @@ -942,7 +858,7 @@ def combine_to_movie(self) -> None: # determine output path movie_file_path = self.movie_file_path - if is_gif_format(): + if self.output_spec.is_gif: movie_file_path = self.gif_file_path if len(partial_movie_files) == 0: # Prevent calling concat on empty list @@ -950,15 +866,16 @@ def combine_to_movie(self) -> None: return logger.info("Combining to Movie file.") + self._write_concat_manifest(partial_movie_files) self.combine_files( partial_movie_files, movie_file_path, - is_gif_format(), + self.output_spec.is_gif, self.includes_sound, ) # handle sound - if self.includes_sound and config.format != "gif": + if self.includes_sound and not self.output_spec.is_gif: sound_file_path = movie_file_path.with_suffix(".wav") # Makes sure sound file length will match video file self.add_audio_segment(AudioSegment.silent(0)) @@ -973,11 +890,11 @@ def combine_to_movie(self) -> None: # but tries to call ffmpeg via its CLI -- which we want # to avoid. This is why we need to do the conversion # manually. - if config.movie_file_extension == ".webm": + if self.output_spec.segment_extension == ".webm": ogg_sound_file_path = sound_file_path.with_suffix(".ogg") convert_audio(sound_file_path, ogg_sound_file_path, "libvorbis") sound_file_path = ogg_sound_file_path - elif config.movie_file_extension == ".mp4": + elif self.output_spec.segment_extension == ".mp4": # Similarly, pyav may reject wav audio in an .mp4 file; # convert to AAC. aac_sound_file_path = sound_file_path.with_suffix(".aac") @@ -1032,7 +949,7 @@ def combine_to_movie(self) -> None: sound_file_path.unlink() self.print_file_ready_message(str(movie_file_path)) - if write_to_movie(): + if self.output_spec.is_video: for file_path in partial_movie_files: # We have to modify the accessed time so if we have to clean the cache we remove the one used the longest. modify_atime(file_path) @@ -1045,78 +962,29 @@ def combine_to_section_videos(self) -> None: # only if section does want to be saved if section.video is not None: logger.info(f"Combining partial files for section '{section.name}'") + section_path = self.sections_output_dir / section.video self.combine_files( section.get_clean_partial_movie_files(), - self.sections_output_dir / section.video, + section_path, ) sections_index.append(section.get_dict(self.sections_output_dir)) - with (self.sections_output_dir / f"{self.output_name}.json").open("w") as file: + section_index = self.output_plan.section_index + assert section_index is not None + section_index.parent.mkdir(parents=True, exist_ok=True) + with section_index.open("w") as file: json.dump(sections_index, file, indent=4) - def _cached_partial_movie_files(self) -> list[Path]: - """Return the partial movie files currently contained in the cache. - - The partial movie file list is excluded (matching by file name: a - ``Path`` never compares equal to its ``str`` name). Hidden files are - excluded as well: on macOS, Finder leaves resource forks (``._*.mp4``) - and ``.DS_Store`` files in the directory which must not count against - ``max_files_cached`` and may vanish again before they could be - deleted. - """ - return [ - self.partial_movie_directory / file_name - for file_name in self.partial_movie_directory.iterdir() - if file_name.name != "partial_movie_file_list.txt" - and not file_name.name.startswith(".") - ] - - def clean_cache(self) -> None: - """Will clean the cache by removing the oldest partial_movie_files.""" - cached_partial_movies = self._cached_partial_movie_files() - if len(cached_partial_movies) > config["max_files_cached"]: - number_files_to_delete = ( - len(cached_partial_movies) - config["max_files_cached"] - ) - - def access_time(path: Path) -> float: - try: - return path.stat().st_atime - except FileNotFoundError: - # The file vanished between listing the directory and - # now; sort it first and let unlink(missing_ok=True) - # handle its deletion without evicting an existing file. - return float("-inf") - - oldest_files_to_delete = sorted( - cached_partial_movies, - key=access_time, - )[:number_files_to_delete] - for file_to_delete in oldest_files_to_delete: - file_to_delete.unlink(missing_ok=True) - logger.info( - f"The partial movie directory is full (> {config['max_files_cached']} files). Therefore, manim has removed the {number_files_to_delete} oldest file(s)." - " You can change this behaviour by changing max_files_cached in config.", - ) - - def flush_cache_directory(self) -> None: - """Delete all the cached partial movie files""" - cached_partial_movies = self._cached_partial_movie_files() - for f in cached_partial_movies: - f.unlink(missing_ok=True) - logger.info( - f"Cache flushed. {len(cached_partial_movies)} file(s) deleted in %(par_dir)s.", - {"par_dir": self.partial_movie_directory}, - ) - def write_subcaption_file(self) -> None: - """Writes the subcaption file.""" - if config.output_file is None: + """Writes the subcaption file next to the primary video artifact.""" + if not self.output_spec.is_video: return - subcaption_file = Path(config.output_file).with_suffix(".srt") + subcaption_file = self.output_plan.subcaption_file + assert subcaption_file is not None + subcaption_file.parent.mkdir(parents=True, exist_ok=True) subcaption_file.write_text(srt.compose(self.subcaptions), encoding="utf-8") logger.info(f"Subcaption file has been written as {subcaption_file}") def print_file_ready_message(self, file_path: StrPath) -> None: - """Prints the "File Ready" message to STDOUT.""" - config["output_file"] = file_path + """Record and report a completed primary artifact.""" + self.final_file_path = Path(file_path) logger.info("\nFile ready at %(file_path)s\n", {"file_path": f"'{file_path}'"}) diff --git a/manim/scene/video_segment_encoder.py b/manim/scene/video_segment_encoder.py new file mode 100644 index 0000000000..ffc5ec9d58 --- /dev/null +++ b/manim/scene/video_segment_encoder.py @@ -0,0 +1,139 @@ +"""Encoding for one silent cached video segment.""" + +from __future__ import annotations + +from contextlib import suppress +from fractions import Fraction +from pathlib import Path + +import av +import numpy as np + +from manim._config.video_encoder import VideoEncoderSpec +from manim.typing import RGBAPixelArray + +__all__ = ["VideoSegmentEncoder"] + + +class VideoSegmentEncoder: + """Encode top-left-origin C-contiguous RGBA frames into one video segment.""" + + def __init__(self, *, target: Path, spec: VideoEncoderSpec) -> None: + self.target = target + self.spec = spec + self._next_pts = 0 + self._closed = False + self.target.parent.mkdir(parents=True, exist_ok=True) + + container = None + try: + container = av.open( + self.target, + mode="w", + format=self.spec.container_format, + ) + stream = container.add_stream( + self.spec.codec, + rate=self.spec.frame_rate, + options=dict(self.spec.options), + ) + stream.pix_fmt = self.spec.pixel_format + stream.width = self.spec.width + stream.height = self.spec.height + except BaseException as error: + if container is not None: + with suppress(BaseException): + container.close() + with suppress(OSError): + self.target.unlink(missing_ok=True) + raise self._operation_error("open", error) from error + + self._container = container + self._stream = stream + + def _profile_description(self) -> str: + return ( + f"{self.spec.container_format}/{self.spec.codec}/" + f"{self.spec.pixel_format}, {self.spec.width}x{self.spec.height} " + f"at {self.spec.frame_rate} fps" + ) + + def _operation_error(self, operation: str, error: BaseException) -> RuntimeError: + return RuntimeError( + f"Failed to {operation} video segment {self.target} " + f"({self._profile_description()}): {error}", + ) + + def _validate_frame(self, pixels: RGBAPixelArray, repeat: int) -> None: + if self._closed: + raise RuntimeError(f"Video segment encoder for {self.target} is closed.") + if isinstance(repeat, bool) or not isinstance(repeat, int) or repeat <= 0: + raise ValueError("Frame repeat must be a positive integer.") + if not isinstance(pixels, np.ndarray): + raise TypeError("Video segment frames must be NumPy arrays.") + expected_shape = (self.spec.height, self.spec.width, 4) + if pixels.shape != expected_shape: + raise ValueError( + f"Video segment frames must have shape {expected_shape}; " + f"got {pixels.shape}.", + ) + if pixels.dtype != np.uint8: + raise TypeError( + f"Video segment frames must use uint8; got {pixels.dtype}.", + ) + if not pixels.flags.c_contiguous: + raise ValueError("Video segment frames must be C-contiguous.") + + def write_frame(self, pixels: RGBAPixelArray, *, repeat: int = 1) -> None: + """Encode ``pixels`` at consecutive segment-local PTS values.""" + self._validate_frame(pixels, repeat) + time_base = Fraction( + self.spec.frame_rate.denominator, + self.spec.frame_rate.numerator, + ) + try: + for _ in range(repeat): + frame = av.VideoFrame.from_ndarray(pixels, format="rgba") + frame.pts = self._next_pts + frame.time_base = time_base + self._next_pts += 1 + for packet in self._stream.encode(frame): + self._container.mux(packet) + except BaseException as error: + raise self._operation_error("encode", error) from error + + def finish(self) -> None: + """Flush encoded packets and close the segment.""" + if self._closed: + return + self._closed = True + first_error: BaseException | None = None + try: + for packet in self._stream.encode(): + self._container.mux(packet) + except BaseException as error: + first_error = error + try: + self._container.close() + except BaseException as error: + if first_error is None: + first_error = error + if first_error is not None: + raise self._operation_error("finish", first_error) from first_error + + def abort(self) -> None: + """Close resources and remove the incomplete target.""" + first_error: BaseException | None = None + if not self._closed: + self._closed = True + try: + self._container.close() + except BaseException as error: + first_error = error + try: + self.target.unlink(missing_ok=True) + except BaseException as error: + if first_error is None: + first_error = error + if first_error is not None: + raise self._operation_error("abort", first_error) from first_error diff --git a/manim/utils/caching.py b/manim/utils/caching.py index 9ab7e6bfd0..78e011071a 100644 --- a/manim/utils/caching.py +++ b/manim/utils/caching.py @@ -1,12 +1,82 @@ from __future__ import annotations from collections.abc import Callable +from pathlib import Path from typing import TYPE_CHECKING, Any from .. import config, logger +from .._config.video_encoder import video_encoder_fingerprint from ..utils.hashing import get_hash_from_play_call -__all__ = ["handle_caching_play"] +__all__ = [ + "clear_segment_cache", + "handle_caching_play", + "prune_segment_cache", +] + +_SEGMENT_EXTENSIONS = frozenset({".mov", ".mp4", ".webm"}) + + +def _segment_cache_files(directory: Path) -> list[Path]: + """Return recognized segment files currently present in ``directory``.""" + try: + entries = directory.iterdir() + return [ + entry + for entry in entries + if not entry.name.startswith(".") + and entry.suffix.lower() in _SEGMENT_EXTENSIONS + and entry.is_file() + ] + except FileNotFoundError: + return [] + + +def prune_segment_cache(directory: Path, max_files: int) -> None: + """Remove the least recently accessed segments beyond ``max_files``. + + ``max_files=-1`` leaves the cache unlimited. Files that disappear during + pruning count toward the number selected for removal, preventing a race + from evicting an additional live segment. + """ + if max_files == -1: + return + if max_files < 0: + raise ValueError("max_files must be non-negative or -1 for unlimited") + + cached_segments = _segment_cache_files(directory) + excess = len(cached_segments) - max_files + if excess <= 0: + return + + def access_time(path: Path) -> float: + try: + return path.stat().st_atime + except FileNotFoundError: + return float("-inf") + + for segment in sorted(cached_segments, key=access_time)[:excess]: + segment.unlink(missing_ok=True) + + logger.info( + "The segment cache exceeded %d files; removed %d least recently used " + "segment(s). Change max_files_cached to adjust this limit.", + max_files, + excess, + ) + + +def clear_segment_cache(directory: Path) -> int: + """Delete recognized segment files from ``directory`` and return the count.""" + removed = 0 + for segment in _segment_cache_files(directory): + try: + segment.unlink() + except FileNotFoundError: + continue + removed += 1 + return removed + if TYPE_CHECKING: from manim.renderer.opengl_renderer import OpenGLRenderer @@ -53,6 +123,15 @@ def wrapper(self: OpenGLRenderer, scene: Scene, *args: Any, **kwargs: Any) -> No self.camera, animations, mobjects_on_scene, + backend="opengl", + encoder_fingerprint=video_encoder_fingerprint( + scene.session_spec.video_encoder, + ), + renderer_state={ + "meshes": scene.meshes, + "background_color": self.background_color, + "anti_alias_width": self.anti_alias_width, + }, ) if self.file_writer.is_already_cached(hash_play): logger.info( diff --git a/manim/utils/docbuild/manim_directive.py b/manim/utils/docbuild/manim_directive.py index bf4fb554ce..fa5e9c26ef 100644 --- a/manim/utils/docbuild/manim_directive.py +++ b/manim/utils/docbuild/manim_directive.py @@ -282,11 +282,10 @@ def run(self) -> list[nodes.Element]: "pixel_height": pixel_height, "pixel_width": pixel_width, "save_last_frame": save_last_frame, - "write_to_movie": not save_last_frame, "output_file": output_file, } if save_last_frame: - example_config["format"] = None + example_config["format"] = "png" if save_as_gif: example_config["format"] = "gif" @@ -299,15 +298,21 @@ def run(self) -> list[nodes.Element]: code = [ "from manim import *", *user_code, - f"{clsname}().render()", + f"_manim_rendered_scene = {clsname}()", + "_manim_rendered_scene.render()", ] + render_namespace = globals() try: with tempconfig(example_config): - run_time = timeit(lambda: exec("\n".join(code), globals()), number=1) - video_dir = config.get_dir("video_dir") - images_dir = config.get_dir("images_dir") + run_time = timeit( + lambda: exec("\n".join(code), render_namespace), + number=1, + ) + rendered_scene = render_namespace.pop("_manim_rendered_scene") + filesrc = rendered_scene.renderer.file_writer.final_file_path except Exception as e: + render_namespace.pop("_manim_rendered_scene", None) raise RuntimeError(f"Error while rendering example {clsname}") from e _write_rendering_stats( @@ -318,18 +323,9 @@ def run(self) -> list[nodes.Element]: # copy video file to output directory if not (save_as_gif or save_last_frame): - filename = f"{output_file}.mp4" - filesrc = video_dir / filename + filename = filesrc.name destfile = Path(dest_dir, filename) shutil.copyfile(filesrc, destfile) - elif save_as_gif: - filename = f"{output_file}.gif" - filesrc = video_dir / filename - elif save_last_frame: - filename = f"{output_file}.png" - filesrc = images_dir / filename - else: - raise ValueError("Invalid combination of render flags received.") rendered_template = jinja2.Template(TEMPLATE).render( clsname=clsname, clsname_lowercase=clsname.lower(), diff --git a/manim/utils/file_ops.py b/manim/utils/file_ops.py index f94a076184..7bf291cf0e 100644 --- a/manim/utils/file_ops.py +++ b/manim/utils/file_ops.py @@ -9,12 +9,6 @@ "seek_full_path_from_defaults", "modify_atime", "open_file", - "is_mp4_format", - "is_gif_format", - "is_png_format", - "is_webm_format", - "is_mov_format", - "write_to_movie", "ensure_executable", ] @@ -37,98 +31,6 @@ from .. import console -def is_mp4_format() -> bool: - """ - Determines if output format is .mp4 - - Returns - ------- - class:`bool` - ``True`` if format is set as mp4 - - """ - val: bool = config["format"] == "mp4" - return val - - -def is_gif_format() -> bool: - """ - Determines if output format is .gif - - Returns - ------- - class:`bool` - ``True`` if format is set as gif - - """ - val: bool = config["format"] == "gif" - return val - - -def is_webm_format() -> bool: - """ - Determines if output format is .webm - - Returns - ------- - class:`bool` - ``True`` if format is set as webm - - """ - val: bool = config["format"] == "webm" - return val - - -def is_mov_format() -> bool: - """ - Determines if output format is .mov - - Returns - ------- - class:`bool` - ``True`` if format is set as mov - - """ - val: bool = config["format"] == "mov" - return val - - -def is_png_format() -> bool: - """ - Determines if output format is .png - - Returns - ------- - class:`bool` - ``True`` if format is set as png - - """ - val: bool = config["format"] == "png" - return val - - -def write_to_movie() -> bool: - """ - Determines from config if the output is a video format such as mp4 or gif, if the --format is set as 'png' - then it will take precedence event if the write_to_movie flag is set - - Returns - ------- - class:`bool` - ``True`` if the output should be written in a movie format - - """ - if is_png_format(): - return False - return ( - config["write_to_movie"] - or is_mp4_format() - or is_gif_format() - or is_webm_format() - or is_mov_format() - ) - - def ensure_executable(path_to_exe: Path) -> bool: if path_to_exe.parent == Path("."): executable: StrPath | None = shutil.which(path_to_exe.stem) @@ -220,20 +122,22 @@ def open_file(file_path: Path, in_browser: bool = False) -> None: sp.run(commands) -def open_media_file(file_writer: SceneFileWriter) -> None: - file_paths = [] - - if config["save_last_frame"]: - file_paths.append(file_writer.image_file_path) - if write_to_movie() and not is_gif_format(): - file_paths.append(file_writer.movie_file_path) - if write_to_movie() and is_gif_format(): - file_paths.append(file_writer.gif_file_path) +def open_media_file( + file_writer: SceneFileWriter, + *, + preview: bool, + show_in_file_browser: bool, +) -> None: + final_file_path = getattr(file_writer, "final_file_path", None) + if final_file_path is None: + logger.warning("No media artifact is available to open.") + return + file_paths = [final_file_path] for file_path in file_paths: - if config["show_in_file_browser"]: + if show_in_file_browser: open_file(file_path, True) - if config["preview"]: + if preview: open_file(file_path, False) logger.info(f"Previewed File at: '{file_path}'") diff --git a/manim/utils/hashing.py b/manim/utils/hashing.py index 4dfd11c1da..3b36e00176 100644 --- a/manim/utils/hashing.py +++ b/manim/utils/hashing.py @@ -418,8 +418,12 @@ def get_hash_from_play_call( camera_object: Camera | OpenGLCamera, animations_list: Iterable[Animation], current_mobjects_list: Iterable[Mobject], + *, + backend: str, + encoder_fingerprint: str, + renderer_state: Any, ) -> str: - """Take the list of animations and a list of mobjects and output their hashes. This is meant to be used for `scene.play` function. + """Return the visual-segment cache key for one compiled play call. Parameters ----------- @@ -434,12 +438,23 @@ def get_hash_from_play_call( current_mobjects_list The list of mobjects. + backend + Stable identity of the renderer producing the segment. + encoder_fingerprint + Stable identity of the resolved segment encoder settings. + renderer_state + Additional renderer-specific state which affects segment pixels. Returns ------- :class:`str` - A string concatenation of the respective hashes of `camera_object`, `animations_list` and `current_mobjects_list`, separated by `_`. + A filename-safe digest of all visual cache inputs. """ + if backend not in {"cairo", "opengl"}: + raise ValueError(f"Unsupported cache backend: {backend}") + if not encoder_fingerprint: + raise ValueError("An encoder fingerprint is required for cache identity.") + logger.debug("Hashing ...") t_start = perf_counter() memoizer = _Memoizer() @@ -453,12 +468,32 @@ def get_hash_from_play_call( _get_json(mobject, memoizer, include_pixel_array=True) for mobject in current_mobjects_list ] - hash_camera, hash_animations, hash_current_mobjects = ( - zlib.crc32(repr(json_val).encode()) - for json_val in [camera_json, animations_list_json, current_mobjects_list_json] + renderer_state_json = _get_json( + renderer_state, + memoizer, + include_pixel_array=True, ) - hash_complete = f"{hash_camera}_{hash_animations}_{hash_current_mobjects}" + digest = hashlib.sha256() + + def feed(value: str) -> None: + encoded = value.encode("utf-8") + digest.update(len(encoded).to_bytes(8, "big")) + digest.update(encoded) + + for name, values in ( + ("backend", [backend]), + ("encoder", [encoder_fingerprint]), + ("camera", [camera_json]), + ("animations", animations_list_json), + ("mobjects", current_mobjects_list_json), + ("renderer", [renderer_state_json]), + ): + feed(name) + feed(str(len(values))) + for value in values: + feed(value) + cache_key = digest.hexdigest() t_end = perf_counter() logger.debug("Hashing done in %(time)s s.", {"time": str(t_end - t_start)[:8]}) - logger.debug("Hash generated : %(h)s", {"h": hash_complete}) - return hash_complete + logger.debug("Hash generated : %(h)s", {"h": cache_key}) + return cache_key diff --git a/manim/utils/ipython_magic.py b/manim/utils/ipython_magic.py index 1d62bbc6f4..cb3040906b 100644 --- a/manim/utils/ipython_magic.py +++ b/manim/utils/ipython_magic.py @@ -150,11 +150,19 @@ def construct(self): if renderer is not None and renderer.window is not None: renderer.window.close() - if config["output_file"] is None: + output_file = getattr( + scene.renderer.file_writer, + "final_file_path", + None, + ) + if output_file is None: logger.info("No output file produced") return + if output_file.is_dir(): + logger.info("Image-sequence output is not displayed in notebooks") + return - local_path = Path(config["output_file"]).relative_to(Path.cwd()) + local_path = output_file.relative_to(Path.cwd()) tmpfile = ( Path(config["media_dir"]) / "jupyter" @@ -167,7 +175,7 @@ def construct(self): tmpfile.parent.mkdir(parents=True, exist_ok=True) shutil.copy(local_path, tmpfile) - file_type = mimetypes.guess_type(config["output_file"])[0] + file_type = mimetypes.guess_type(output_file)[0] assert isinstance(file_type, str) embed = config["media_embed"] if not embed: @@ -177,7 +185,7 @@ def construct(self): embed = "google.colab" in str(get_ipython()) if file_type.startswith("image"): - result = Image(filename=config["output_file"]) + result = Image(filename=output_file) else: result = Video( tmpfile, diff --git a/manim/utils/module_ops.py b/manim/utils/module_ops.py index e4c9374403..ddca750b2f 100644 --- a/manim/utils/module_ops.py +++ b/manim/utils/module_ops.py @@ -16,7 +16,6 @@ NO_SCENE_MESSAGE, SCENE_NOT_FOUND_MESSAGE, ) -from manim.scene.scene_file_writer import SceneFileWriter if TYPE_CHECKING: from manim.scene.scene import Scene @@ -112,7 +111,6 @@ def get_scenes_to_render(scene_classes: list[type[Scene]]) -> list[type[Scene]]: def prompt_user_for_choice(scene_classes: list[type[Scene]]) -> list[type[Scene]]: num_to_class = {} - SceneFileWriter.force_output_as_scene_name = True for count, scene_class in enumerate(scene_classes, 1): name = scene_class.__name__ console.print(f"{count}: {name}", style="logging.level.info") diff --git a/manim/utils/sounds.py b/manim/utils/sounds.py index de7b6be80e..ae6c909f2d 100644 --- a/manim/utils/sounds.py +++ b/manim/utils/sounds.py @@ -8,7 +8,6 @@ from typing import TYPE_CHECKING -from .. import config from ..utils.file_ops import seek_full_path_from_defaults if TYPE_CHECKING: @@ -18,9 +17,10 @@ # Still in use by add_sound() function in scene_file_writer.py -def get_full_sound_file_path(sound_file_name: StrPath) -> Path: +def get_full_sound_file_path(sound_file_name: StrPath, assets_dir: Path) -> Path: + """Locate a sound path directly or relative to ``assets_dir``.""" return seek_full_path_from_defaults( sound_file_name, - default_dir=config.get_dir("assets_dir"), + default_dir=assets_dir, extensions=[".wav", ".mp3"], ) diff --git a/manim/utils/testing/_test_class_makers.py b/manim/utils/testing/_test_class_makers.py index b7b53306d3..49a1a85148 100644 --- a/manim/utils/testing/_test_class_makers.py +++ b/manim/utils/testing/_test_class_makers.py @@ -6,7 +6,7 @@ from manim.renderer.cairo_renderer import CairoRenderer from manim.renderer.opengl_renderer import OpenGLRenderer from manim.scene.scene import Scene -from manim.scene.scene_file_writer import SceneFileWriter +from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings from manim.typing import PixelArray, StrPath from ._frames_testers import _FramesTester @@ -44,23 +44,19 @@ class _TestRenderer(from_renderer): class DummySceneFileWriter(SceneFileWriter): """Delegate of SceneFileWriter used to test the frames.""" - def __init__( - self, - renderer: CairoRenderer | OpenGLRenderer, - scene_name: str, - **kwargs: Any, - ) -> None: - super().__init__(renderer, scene_name, **kwargs) + def __init__(self, settings: _SceneFileWriterSettings) -> None: + super().__init__(settings) self.i = 0 - def init_output_directories(self, scene_name: str) -> None: - pass - def add_partial_movie_file(self, hash_animation: str | None) -> None: pass def begin_animation( - self, allow_write: bool = True, file_path: StrPath | None = None + self, + allow_write: bool = True, + *, + animation_index: int, + file_path: StrPath | None = None, ) -> Any: pass @@ -73,21 +69,14 @@ def combine_to_movie(self) -> None: def combine_to_section_videos(self) -> None: pass - def clean_cache(self) -> None: - pass - - def write_frame( - self, frame_or_renderer: PixelArray | OpenGLRenderer, num_frames: int = 1 - ) -> None: + def write_frame(self, pixels: PixelArray, *, repeat: int = 1) -> None: self.i += 1 def _make_scene_file_writer_class(tester: _FramesTester) -> type[SceneFileWriter]: class TestSceneFileWriter(DummySceneFileWriter): - def write_frame( - self, frame_or_renderer: PixelArray | OpenGLRenderer, num_frames: int = 1 - ) -> None: - tester.check_frame(self.i, frame_or_renderer) - super().write_frame(frame_or_renderer, num_frames=num_frames) + def write_frame(self, pixels: PixelArray, *, repeat: int = 1) -> None: + tester.check_frame(self.i, pixels) + super().write_frame(pixels, repeat=repeat) return TestSceneFileWriter diff --git a/tests/control_data/videos_data/SceneWithSections.json b/tests/control_data/videos_data/SceneWithSections.json index 5278526e57..1d9e8df912 100644 --- a/tests/control_data/videos_data/SceneWithSections.json +++ b/tests/control_data/videos_data/SceneWithSections.json @@ -12,7 +12,7 @@ "section_dir_layout": [ "SceneWithSections.json", "SceneWithSections_0004_unnamed.mp4", - "SceneWithSections_0003_Prepare For Unforeseen Consequences..mp4", + "SceneWithSections_0003_Prepare-For-Unforeseen-Consequences.mp4", "SceneWithSections_0002_test.mp4", "SceneWithSections_0001_unnamed.mp4", "SceneWithSections_0000_autocreated.mp4", @@ -58,7 +58,7 @@ { "name": "Prepare For Unforeseen Consequences.", "type": "default.normal", - "video": "SceneWithSections_0003_Prepare For Unforeseen Consequences..mp4", + "video": "SceneWithSections_0003_Prepare-For-Unforeseen-Consequences.mp4", "codec_name": "h264", "width": 854, "height": 480, diff --git a/tests/control_data/videos_data/SceneWithSkipAnimations.json b/tests/control_data/videos_data/SceneWithSkipAnimations.json index 71bb21abb3..2470682da3 100644 --- a/tests/control_data/videos_data/SceneWithSkipAnimations.json +++ b/tests/control_data/videos_data/SceneWithSkipAnimations.json @@ -11,16 +11,16 @@ }, "section_dir_layout": [ "ElaborateSceneWithSections.json", - "ElaborateSceneWithSections_0003_fade out.mp4", - "ElaborateSceneWithSections_0001_transform to circle.mp4", - "ElaborateSceneWithSections_0000_create square.mp4", + "ElaborateSceneWithSections_0003_fade-out.mp4", + "ElaborateSceneWithSections_0001_transform-to-circle.mp4", + "ElaborateSceneWithSections_0000_create-square.mp4", "." ], "section_index": [ { "name": "create square", "type": "default.normal", - "video": "ElaborateSceneWithSections_0000_create square.mp4", + "video": "ElaborateSceneWithSections_0000_create-square.mp4", "codec_name": "h264", "width": 854, "height": 480, @@ -32,7 +32,7 @@ { "name": "transform to circle", "type": "default.normal", - "video": "ElaborateSceneWithSections_0001_transform to circle.mp4", + "video": "ElaborateSceneWithSections_0001_transform-to-circle.mp4", "codec_name": "h264", "width": 854, "height": 480, @@ -44,7 +44,7 @@ { "name": "fade out", "type": "default.normal", - "video": "ElaborateSceneWithSections_0003_fade out.mp4", + "video": "ElaborateSceneWithSections_0003_fade-out.mp4", "codec_name": "h264", "width": 854, "height": 480, diff --git a/tests/helpers/graphical_units.py b/tests/helpers/graphical_units.py index 232a6510eb..9438f930e7 100644 --- a/tests/helpers/graphical_units.py +++ b/tests/helpers/graphical_units.py @@ -31,7 +31,7 @@ def set_test_scene(scene_object: type[Scene], module_name: str, config): set_test_scene(DotTest, "geometry") """ - config["write_to_movie"] = False + config.format = "none" config["disable_caching"] = True config["format"] = "png" config["pixel_height"] = 480 diff --git a/tests/interface/test_cache_command.py b/tests/interface/test_cache_command.py new file mode 100644 index 0000000000..177eb8b02e --- /dev/null +++ b/tests/interface/test_cache_command.py @@ -0,0 +1,116 @@ +from __future__ import annotations + +from pathlib import Path + +import pytest +from click.testing import CliRunner + +from manim.__main__ import main + + +def _segment_directory( + media_dir: Path, + *, + module_name: str, + quality: str, + scene_name: str, +) -> Path: + return ( + media_dir + / "videos" + / module_name + / quality + / "partial_movie_files" + / scene_name + ) + + +def test_cache_clear_resolves_each_scene_without_importing_file(tmp_path): + scene_file = tmp_path / "example.py" + scene_file.write_text("raise RuntimeError('must not be imported')\n") + media_dir = tmp_path / "media" + + for scene_name in ("FirstScene", "SecondScene"): + directory = _segment_directory( + media_dir, + module_name="example", + quality="480p15", + scene_name=scene_name, + ) + directory.mkdir(parents=True) + (directory / "segment.mp4").touch() + + result = CliRunner().invoke( + main, + [ + "cache", + "clear", + "--media-dir", + str(media_dir), + "-q", + "l", + str(scene_file), + "FirstScene", + "SecondScene", + ], + prog_name="manim", + ) + + assert result.exception is None, result.output + for scene_name in ("FirstScene", "SecondScene"): + directory = _segment_directory( + media_dir, + module_name="example", + quality="480p15", + scene_name=scene_name, + ) + assert not (directory / "segment.mp4").exists() + assert "Removed 1 cached segment(s) for FirstScene" in result.output + assert "Removed 1 cached segment(s) for SecondScene" in result.output + + +@pytest.mark.parametrize("override_media_dir", [False, True]) +def test_cache_clear_applies_config_file_and_path_overrides( + tmp_path, + override_media_dir, +): + scene_file = tmp_path / "configured_scene.py" + scene_file.touch() + configured_media_dir = tmp_path / "configured-media" + selected_media_dir = ( + tmp_path / "overridden-media" if override_media_dir else configured_media_dir + ) + config_file = tmp_path / "cache.cfg" + config_file.write_text(f"[CLI]\nmedia_dir = {configured_media_dir}\n") + + directory = _segment_directory( + selected_media_dir, + module_name="configured_scene", + quality="360p12", + scene_name="ConfiguredScene", + ) + directory.mkdir(parents=True) + segment = directory / "segment.mp4" + segment.touch() + + options = ["--config-file", str(config_file)] + if override_media_dir: + options.extend(["--media-dir", str(selected_media_dir)]) + result = CliRunner().invoke( + main, + [ + "cache", + "clear", + *options, + "--resolution", + "640,360", + "--fps", + "12", + str(scene_file), + "ConfiguredScene", + ], + prog_name="manim", + ) + + assert result.exception is None, result.output + assert not segment.exists() diff --git a/tests/module/test_manager.py b/tests/module/test_manager.py index df85ae6f47..5423126979 100644 --- a/tests/module/test_manager.py +++ b/tests/module/test_manager.py @@ -3,14 +3,16 @@ import copy import datetime import threading -from unittest.mock import Mock +from unittest.mock import Mock, patch import pytest import srt from manim import Manager, Scene, tempconfig +from manim._config.output import OutputFormat from manim.animation.animation import Wait from manim.constants import RendererType +from manim.renderer.protocol import RendererCapabilities from manim.scene.scene import SceneInteractRerun from manim.utils.exceptions import EndSceneEarlyException, RerunSceneException @@ -24,6 +26,82 @@ def test_manager_attaches_to_existing_scene(dry_run): assert scene.manager is manager +def test_manager_exposes_the_session_output_snapshot(config): + config.dry_run = False + config.format = "gif" + config.preview = True + scene = Scene() + manager = Manager(scene) + + config.format = "none" + config.preview = False + config.dry_run = True + + assert manager.output_spec.format is OutputFormat.GIF + assert manager.output_spec is scene.renderer.file_writer.output_spec + assert manager.session_spec.presentation.open_after_render is True + assert manager.session_spec.dry_run is False + + +def test_manager_exposes_dry_run_session_intent(config): + config.format = "gif" + config.dry_run = True + scene = Scene() + manager = Manager(scene) + + config.dry_run = False + + assert manager.output_spec.format is OutputFormat.NONE + assert manager.session_spec.dry_run is True + + +def test_post_render_preview_requires_an_artifact(dry_run): + scene = Scene() + + with pytest.raises(ValueError, match="requires a media artifact"): + Manager(scene).render(preview=True) + + +@pytest.mark.parametrize("output_format", ["mp4", "mov", "webm", "gif"]) +def test_manager_rejects_explicit_video_for_scene_without_play_calls( + config, + output_format, +): + config.format = output_format + renderer = Mock() + renderer.capabilities = RendererCapabilities() + renderer.num_plays = 0 + scene = Scene(renderer) + manager = Manager(scene) + + with pytest.raises( + RuntimeError, + match=f"explicitly requested {output_format.upper()}", + ): + manager.post_construct() + + renderer.scene_finished.assert_not_called() + + +def test_manager_warns_when_automatic_video_falls_back_to_still(config): + config.format = "auto" + renderer = Mock() + renderer.capabilities = RendererCapabilities() + renderer.num_plays = 0 + scene = Scene(renderer) + renderer.file_writer.final_file_path = "scene.png" + manager = Manager(scene) + + with patch("manim.manager.logger.warning") as warning: + manager.post_construct() + + renderer.scene_finished.assert_called_once_with(scene) + warning.assert_called_once_with( + f"{scene} has no play calls. Automatic video output has been saved as a " + "PNG instead.", + ) + + def test_manager_rejects_second_attachment(dry_run): scene = Scene() manager = Manager(scene) diff --git a/tests/module/test_output_path_behavior.py b/tests/module/test_output_path_behavior.py new file mode 100644 index 0000000000..9f68c75f10 --- /dev/null +++ b/tests/module/test_output_path_behavior.py @@ -0,0 +1,287 @@ +from __future__ import annotations + +from pathlib import Path +from unittest.mock import Mock + +import numpy as np +import pytest + +from manim import __version__ +from manim._config.output import OutputFormat, OutputSpec +from manim._config.output_plan import ( + resolve_media_layout, + resolve_module_name, + resolve_output_plan, + resolve_requested_output_name, +) +from manim._config.video_encoder import resolve_video_encoder +from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings + + +def _make_writer( + config, + tmp_path: Path, + output_format: OutputFormat, + *, + transparent: bool = False, + save_sections: bool = False, + fallback_to_still: bool = False, + output_file: str | Path = "", +) -> SceneFileWriter: + config.media_dir = tmp_path + config.input_file = tmp_path / "nested" / "example.scene.py" + config.pixel_height = 480 + config.frame_rate = 15 + config.output_file = output_file + + output = OutputSpec( + output_format, + transparent, + save_sections, + fallback_to_still, + ) + module_name = resolve_module_name(config) + layout = resolve_media_layout( + config, + output, + module_name=module_name, + scene_name="ExampleScene", + working_directory=Path.cwd(), + ) + output_plan = resolve_output_plan( + layout, + output, + scene_name="ExampleScene", + requested_output_name=resolve_requested_output_name(config), + ) + settings = _SceneFileWriterSettings( + plan=output_plan, + video_encoder=resolve_video_encoder( + output, + width=config.pixel_width, + height=config.pixel_height, + frame_rate=config.frame_rate, + ), + max_inflight_encoders=config.max_inflight_encoders, + encoder_queue_size=config.encoder_queue_size, + max_files_cached=config.max_files_cached, + assets_dir=Path.cwd(), + ) + return SceneFileWriter(settings) + + +@pytest.mark.parametrize( + ("output_format", "extension"), + [ + (OutputFormat.MP4, ".mp4"), + (OutputFormat.MOV, ".mov"), + (OutputFormat.WEBM, ".webm"), + ], +) +def test_default_video_paths(config, tmp_path, output_format, extension): + writer = _make_writer(config, tmp_path, output_format) + quality_dir = tmp_path / "videos" / "example.scene" / "480p15" + + assert writer.movie_file_path == quality_dir / f"ExampleScene{extension}" + assert writer.partial_movie_directory == ( + quality_dir / "partial_movie_files" / "ExampleScene" + ) + assert writer.output_plan.fallback_image is None + with pytest.raises(AttributeError, match="does not contain an image path"): + writer.image_file_path + + +@pytest.mark.parametrize( + ("transparent", "segment_extension"), + [(False, ".mp4"), (True, ".mov")], +) +def test_gif_primary_and_segment_paths( + config, + tmp_path, + transparent, + segment_extension, +): + writer = _make_writer( + config, + tmp_path, + OutputFormat.GIF, + transparent=transparent, + ) + quality_dir = tmp_path / "videos" / "example.scene" / "480p15" + + assert writer.movie_file_path == ( + quality_dir / f"ExampleScene_ManimCE_v{__version__}.gif" + ) + assert writer.gif_file_path == ( + quality_dir / f"ExampleScene_ManimCE_v{__version__}.gif" + ) + writer.add_partial_movie_file("cache-key") + assert writer.partial_movie_files == [ + str( + quality_dir + / "partial_movie_files" + / "ExampleScene" + / f"cache-key{segment_extension}" + ) + ] + + +def test_default_png_and_automatic_video_fallback_paths(config, tmp_path): + png_writer = _make_writer(config, tmp_path, OutputFormat.PNG) + expected = ( + tmp_path + / "images" + / "example.scene" + / f"ExampleScene_ManimCE_v{__version__}.png" + ) + + png_writer.save_image(np.zeros((1, 1, 4), dtype=np.uint8)) + assert png_writer.final_file_path == expected + + video_writer = _make_writer( + config, + tmp_path, + OutputFormat.MP4, + fallback_to_still=True, + ) + video_writer.save_image(np.zeros((1, 1, 4), dtype=np.uint8)) + assert video_writer.final_file_path == expected + + +def test_png_sequence_path_and_zero_padding(config, tmp_path): + config.zero_pad = 3 + writer = _make_writer(config, tmp_path, OutputFormat.PNG_SEQUENCE) + + expected_dir = tmp_path / "images" / "example.scene" / "ExampleScene" + assert writer.image_sequence_directory == expected_dir + + pixels = np.zeros((1, 1, 4), dtype=np.uint8) + writer.write_frame(pixels) + + assert not hasattr(writer, "renderer") + assert (expected_dir / "000.png").is_file() + + +def test_resolved_output_suffix_preserves_a_different_suffix(config, tmp_path): + matching = _make_writer( + config, + tmp_path, + OutputFormat.MP4, + output_file="movie.mp4", + ) + assert matching.movie_file_path.name == "movie.mp4" + + differing = _make_writer( + config, + tmp_path, + OutputFormat.MP4, + output_file="movie.mov", + ) + assert differing.movie_file_path.name == "movie.mov.mp4" + + +def test_sections_use_configured_directory_for_simple_output_name(config, tmp_path): + writer = _make_writer( + config, + tmp_path, + OutputFormat.MP4, + save_sections=True, + output_file="movie", + ) + writer.next_section("intro", skip_animations=False, type_="default.normal") + + section = writer.sections[-1] + assert writer.sections_output_dir == ( + tmp_path / "videos" / "example.scene" / "480p15" / "sections" + ) + assert section.video == "movie_0000_intro.mp4" + assert writer.sections_output_dir / section.video == ( + writer.sections_output_dir / "movie_0000_intro.mp4" + ) + + +def test_nested_and_absolute_output_names_do_not_relocate_sections( + config, + tmp_path, +): + nested = _make_writer( + config, + tmp_path, + OutputFormat.MP4, + save_sections=True, + output_file="exports/movie", + ) + nested.next_section("intro", skip_animations=False, type_="default.normal") + assert nested.sections_output_dir / nested.sections[-1].video == ( + nested.sections_output_dir / "movie_0000_intro.mp4" + ) + + absolute_name = tmp_path / "exports" / "movie" + absolute = _make_writer( + config, + tmp_path, + OutputFormat.MP4, + save_sections=True, + output_file=absolute_name, + ) + absolute.next_section("intro", skip_animations=False, type_="default.normal") + assert not Path(absolute.sections[-1].video).is_absolute() + assert absolute.sections_output_dir / absolute.sections[-1].video == ( + absolute.sections_output_dir / "movie_0000_intro.mp4" + ) + + +def test_diagnostic_concat_manifest_records_the_full_scene_only( + config, + tmp_path, + monkeypatch, +): + writer = _make_writer( + config, + tmp_path, + OutputFormat.MP4, + save_sections=True, + ) + partial_directory = writer.partial_movie_directory + partial_directory.mkdir(parents=True) + first = partial_directory / "first.mp4" + second = partial_directory / "second.mp4" + first.touch() + second.touch() + writer.partial_movie_files = [str(first), None, str(second)] + + manifest_path = writer.output_plan.concat_manifest + assert manifest_path is not None + manifest_path.write_text("stale") + combine_files = Mock() + monkeypatch.setattr(writer, "combine_files", combine_files) + + writer.combine_to_movie() + + expected_manifest = ( + "# This file records the segment order used by Manim.\n" + f"file 'file:{first.as_posix()}'\n" + f"file 'file:{second.as_posix()}'\n" + ) + assert manifest_path.read_text() == expected_manifest + assert not list(partial_directory.glob(".partial_movie_file_list.txt.*.tmp")) + + section = writer.sections[-1] + section.partial_movie_files = [str(second)] + monkeypatch.setattr( + section, + "get_dict", + Mock(return_value={"name": section.name}), + ) + writer.combine_to_section_videos() + + assert manifest_path.read_text() == expected_manifest + + +def test_no_output_plans_no_media_directories(config, tmp_path): + media_root = tmp_path / "unused-media" + writer = _make_writer(config, media_root, OutputFormat.NONE) + + assert not hasattr(writer, "movie_file_path") + assert not hasattr(writer, "image_file_path") + assert not media_root.exists() diff --git a/tests/module/test_output_plan.py b/tests/module/test_output_plan.py new file mode 100644 index 0000000000..a5c559cfb6 --- /dev/null +++ b/tests/module/test_output_plan.py @@ -0,0 +1,459 @@ +from __future__ import annotations + +from dataclasses import FrozenInstanceError +from pathlib import Path + +import pytest + +from manim import Scene, __version__ +from manim._config.output import OutputFormat, OutputSpec +from manim._config.output_plan import ( + MediaLayoutSpec, + resolve_file_log_path, + resolve_media_layout, + resolve_module_name, + resolve_output_plan, + resolve_requested_output_name, +) +from manim.cli.render.commands import _validate_scene_batch_output_name + + +def _layout(tmp_path: Path, *, sections: bool = False) -> MediaLayoutSpec: + root = tmp_path / "not-created" + return MediaLayoutSpec( + video_dir=root / "videos", + images_dir=root / "images", + sections_dir=root / "sections" if sections else None, + partial_movie_dir=root / "segments", + log_dir=root / "logs", + zero_pad=4, + ) + + +def _output( + output_format: OutputFormat, + *, + transparent: bool = False, + save_sections: bool = False, + fallback_to_still: bool = False, +) -> OutputSpec: + return OutputSpec( + output_format, + transparent, + save_sections, + fallback_to_still, + ) + + +@pytest.mark.parametrize( + ("output_format", "extension"), + [ + (OutputFormat.MP4, ".mp4"), + (OutputFormat.MOV, ".mov"), + (OutputFormat.WEBM, ".webm"), + ], +) +def test_resolve_video_plan(tmp_path, output_format, extension): + layout = _layout(tmp_path) + + output = _output(output_format) + plan = resolve_output_plan( + layout, + output, + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.output is output + assert plan.primary_artifact == layout.video_dir / f"ExampleScene{extension}" + assert plan.fallback_image is None + assert plan.segment_cache_dir == layout.partial_movie_dir + assert plan.segment_path("cache-key") == ( + layout.partial_movie_dir / f"cache-key{extension}" + ) + assert plan.concat_manifest == ( + layout.partial_movie_dir / "partial_movie_file_list.txt" + ) + assert plan.subcaption_file == layout.video_dir / "ExampleScene.srt" + + +@pytest.mark.parametrize( + ("transparent", "segment_extension"), + [(False, ".mp4"), (True, ".mov")], +) +def test_resolve_gif_plan(tmp_path, transparent, segment_extension): + layout = _layout(tmp_path) + + plan = resolve_output_plan( + layout, + _output(OutputFormat.GIF, transparent=transparent), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.primary_artifact == ( + layout.video_dir / f"ExampleScene_ManimCE_v{__version__}.gif" + ) + assert plan.segment_extension == segment_extension + assert plan.segment_path("hash") == layout.partial_movie_dir / ( + f"hash{segment_extension}" + ) + + +def test_resolve_automatic_video_plan_with_fallback(tmp_path): + layout = _layout(tmp_path) + + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4, fallback_to_still=True), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.primary_artifact == layout.video_dir / "ExampleScene.mp4" + assert plan.fallback_image == ( + layout.images_dir / f"ExampleScene_ManimCE_v{__version__}.png" + ) + + +def test_resolve_png_plan(tmp_path): + layout = _layout(tmp_path) + + plan = resolve_output_plan( + layout, + _output(OutputFormat.PNG), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.primary_artifact == ( + layout.images_dir / f"ExampleScene_ManimCE_v{__version__}.png" + ) + assert plan.fallback_image is None + with pytest.raises(ValueError, match="does not contain an image sequence"): + plan.image_frame_path(0) + + +def test_resolve_png_sequence_plan(tmp_path): + layout = _layout(tmp_path) + + plan = resolve_output_plan( + layout, + _output(OutputFormat.PNG_SEQUENCE), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.primary_artifact == layout.images_dir / "ExampleScene" + assert plan.image_sequence_dir == layout.images_dir / "ExampleScene" + assert plan.image_frame_path(0) == layout.images_dir / "ExampleScene" / "0000.png" + assert plan.image_frame_path(42) == ( + layout.images_dir / "ExampleScene" / "0042.png" + ) + + +def test_resolve_no_output_plan_without_layout_directories(tmp_path): + layout = MediaLayoutSpec(None, None, None, None, None, zero_pad=4) + + plan = resolve_output_plan( + layout, + _output(OutputFormat.NONE), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.primary_artifact is None + assert plan.fallback_image is None + assert plan.segment_cache_dir is None + assert plan.image_sequence_dir is None + assert not (tmp_path / "not-created").exists() + + +@pytest.mark.parametrize( + ("requested_name", "expected_name"), + [ + ("movie", "movie.mp4"), + ("movie.mp4", "movie.mp4"), + ("movie.mov", "movie.mov.mp4"), + ], +) +def test_resolved_format_controls_custom_output_suffix( + tmp_path, + requested_name, + expected_name, +): + layout = _layout(tmp_path) + + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4), + scene_name="ExampleScene", + requested_output_name=Path(requested_name), + ) + + assert plan.primary_artifact == layout.video_dir / expected_name + assert plan.output_stem == "movie" + + +def test_absolute_output_name_only_relocates_primary_and_fallback(tmp_path): + layout = _layout(tmp_path, sections=True) + requested = tmp_path / "exports" / "movie.mov" + + plan = resolve_output_plan( + layout, + _output( + OutputFormat.MP4, + save_sections=True, + fallback_to_still=True, + ), + scene_name="ExampleScene", + requested_output_name=requested, + ) + + assert plan.primary_artifact == tmp_path / "exports" / "movie.mov.mp4" + assert plan.fallback_image == tmp_path / "exports" / "movie.mov.png" + assert plan.section_index == layout.sections_dir / "movie.json" + assert plan.section_path(0, "intro") == ( + layout.sections_dir / "movie_0000_intro.mp4" + ) + assert plan.segment_cache_dir == layout.partial_movie_dir + + +def test_nested_output_name_keeps_sections_in_configured_directory(tmp_path): + layout = _layout(tmp_path, sections=True) + + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4, save_sections=True), + scene_name="ExampleScene", + requested_output_name=Path("exports/movie.mp4"), + ) + + assert plan.primary_artifact == layout.video_dir / "exports" / "movie.mp4" + assert plan.section_path(3, "ending") == ( + layout.sections_dir / "movie_0003_ending.mp4" + ) + + +def test_plan_resolution_does_not_create_directories(tmp_path): + layout = _layout(tmp_path, sections=True) + missing_root = tmp_path / "not-created" + + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4, save_sections=True), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.primary_artifact is not None + assert not missing_root.exists() + + +def test_plans_are_immutable_hashable_values(tmp_path): + layout = _layout(tmp_path) + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert hash(layout) + assert hash(plan) + with pytest.raises(FrozenInstanceError): + plan.primary_artifact = tmp_path / "other.mp4" + + +@pytest.mark.parametrize("scene_name", ["", None]) +def test_scene_name_is_required(tmp_path, scene_name): + with pytest.raises(ValueError, match="scene name"): + resolve_output_plan( + _layout(tmp_path), + _output(OutputFormat.MP4), + scene_name=scene_name, + requested_output_name=None, + ) + + +def test_dynamic_path_methods_validate_inputs(tmp_path): + layout = _layout(tmp_path, sections=True) + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4, save_sections=True), + scene_name="ExampleScene", + requested_output_name=None, + ) + + with pytest.raises(ValueError, match="cache key"): + plan.segment_path("../escape") + with pytest.raises(ValueError, match="non-negative"): + plan.section_path(-1, "intro") + with pytest.raises(TypeError, match="strings"): + plan.section_path(0, 1) # type: ignore[arg-type] + + +@pytest.mark.parametrize( + ("name", "slug"), + [ + ("1", "1"), + ("create square", "create-square"), + ("Chapter 1: Why/How?", "Chapter-1-Why-How"), + ("../../../escape", "escape"), + ("Überblick № 2", "Überblick-No-2"), + ("!!!", "section"), + ], +) +def test_section_paths_use_safe_human_readable_slugs(tmp_path, name, slug): + layout = _layout(tmp_path, sections=True) + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4, save_sections=True), + scene_name="ExampleScene", + requested_output_name=None, + ) + + assert plan.section_path(3, name) == ( + layout.sections_dir / f"ExampleScene_0003_{slug}.mp4" + ) + + +def test_section_ordinal_keeps_duplicate_slugs_unique(tmp_path): + layout = _layout(tmp_path, sections=True) + plan = resolve_output_plan( + layout, + _output(OutputFormat.MP4, save_sections=True), + scene_name="ExampleScene", + requested_output_name=None, + ) + + first = plan.section_path(1, "intro!") + second = plan.section_path(2, "intro?") + + assert first.name == "ExampleScene_0001_intro.mp4" + assert second.name == "ExampleScene_0002_intro.mp4" + assert first != second + + +def test_config_adapter_captures_exact_required_directories(config, tmp_path): + config.media_dir = "relative-media" + config.input_file = tmp_path / "source" / "example.py" + config.pixel_height = 480 + config.frame_rate = 15 + config.zero_pad = 3 + config.log_to_file = True + output = _output( + OutputFormat.MP4, + save_sections=True, + fallback_to_still=True, + ) + + module_name = resolve_module_name(config) + layout = resolve_media_layout( + config, + output, + module_name=module_name, + scene_name="ExampleScene", + working_directory=tmp_path, + ) + + quality_dir = tmp_path / "relative-media" / "videos" / "example" / "480p15" + assert module_name == "example" + assert layout.video_dir == quality_dir + assert layout.images_dir == tmp_path / "relative-media" / "images" / "example" + assert layout.sections_dir == quality_dir / "sections" + assert layout.partial_movie_dir == ( + quality_dir / "partial_movie_files" / "ExampleScene" + ) + assert layout.log_dir == tmp_path / "relative-media" / "logs" + assert layout.zero_pad == 3 + assert not (tmp_path / "relative-media").exists() + + +def test_config_adapter_skips_fallback_directory_for_explicit_video( + config, + tmp_path, +): + config.media_dir = "explicit-video" + + layout = resolve_media_layout( + config, + _output(OutputFormat.MP4), + module_name="example", + scene_name="ExampleScene", + working_directory=tmp_path, + ) + + assert layout.images_dir is None + assert layout.video_dir is not None + + +def test_config_adapter_skips_unused_output_directories(config, tmp_path): + config.media_dir = "unused-media" + config.log_to_file = False + + layout = resolve_media_layout( + config, + _output(OutputFormat.NONE), + module_name="", + scene_name="ExampleScene", + working_directory=tmp_path, + ) + + assert layout == MediaLayoutSpec(None, None, None, None, None, zero_pad=4) + + +def test_scene_and_writer_share_immutable_output_plan(config, tmp_path): + initial_media_dir = tmp_path / "initial" + config.media_dir = initial_media_dir + config.input_file = tmp_path / "example.py" + config.format = "mp4" + + scene = Scene() + plan = scene.output_plan + + assert not initial_media_dir.exists() + config.media_dir = tmp_path / "changed" + config.output_file = "changed-name" + + assert scene.renderer.file_writer.output_plan is plan + assert plan.primary_artifact == ( + initial_media_dir / "videos" / "example" / "1080p60" / "Scene.mp4" + ) + + +def test_cli_batch_output_name_validation(config): + config.output_file = "movie" + config.write_all = False + + _validate_scene_batch_output_name([object]) + with pytest.raises(ValueError, match="exactly one scene"): + _validate_scene_batch_output_name([object, object]) + + config.write_all = True + with pytest.raises(ValueError, match="exactly one scene"): + _validate_scene_batch_output_name([object]) + + +def test_requested_name_and_log_path_resolution(config, tmp_path): + config.output_file = "exports/movie.mp4" + config.log_to_file = True + config.media_dir = tmp_path + module_name = resolve_module_name(config) + layout = resolve_media_layout( + config, + _output(OutputFormat.NONE), + module_name=module_name, + scene_name="ExampleScene", + working_directory=tmp_path, + ) + + assert resolve_requested_output_name(config) == Path("exports/movie.mp4") + assert ( + resolve_file_log_path( + layout, + module_name=module_name, + scene_name="ExampleScene", + ) + == tmp_path / "logs" / "_ExampleScene.log" + ) diff --git a/tests/module/test_scene_file_writer_settings.py b/tests/module/test_scene_file_writer_settings.py new file mode 100644 index 0000000000..2d86801205 --- /dev/null +++ b/tests/module/test_scene_file_writer_settings.py @@ -0,0 +1,76 @@ +from dataclasses import replace +from pathlib import Path +from unittest.mock import Mock + +import pytest + +import manim +from manim import Scene, config, tempconfig +from manim.scene import scene_file_writer as writer_module + + +def test_scene_resolves_consistent_writer_settings_with_empty_assets_root(): + with tempconfig( + { + "format": "none", + "assets_dir": "", + "max_inflight_encoders": 3, + "encoder_queue_size": 5, + "max_files_cached": -1, + }, + ): + scene = Scene() + settings = scene.file_writer_settings + + assert scene.renderer.file_writer.settings is settings + assert settings.plan is scene.output_plan + assert settings.plan.output is scene.session_spec.output + assert settings.video_encoder is scene.session_spec.video_encoder + assert settings.max_inflight_encoders == 3 + assert settings.encoder_queue_size == 5 + assert settings.max_files_cached == -1 + assert settings.assets_dir == Path.cwd().absolute() + + +def test_writer_settings_are_private_and_not_exported_from_top_level_manim(): + assert not hasattr(writer_module, "SceneFileWriterSettings") + assert not hasattr(manim, "SceneFileWriterSettings") + assert not hasattr(manim, "_SceneFileWriterSettings") + + +def test_writer_settings_reject_plan_output_mismatches(): + with tempconfig({"format": "mp4"}): + settings = Scene().file_writer_settings + + mismatched_plan = replace(settings.plan, segment_extension=".webm") + with pytest.raises(ValueError, match="plan segment extension"): + replace(settings, plan=mismatched_plan) + + assert settings.video_encoder is not None + mismatched_encoder = replace(settings.video_encoder, container_format="webm") + with pytest.raises(ValueError, match="encoder container"): + replace(settings, video_encoder=mismatched_encoder) + + +def test_writer_uses_captured_assets_root(tmp_path, monkeypatch): + captured_assets = tmp_path / "captured" + changed_assets = tmp_path / "changed" + captured_assets.mkdir() + changed_assets.mkdir() + sound_path = captured_assets / "tone.wav" + sound_path.touch() + + with tempconfig({"format": "none", "assets_dir": captured_assets}): + scene = Scene() + writer = scene.renderer.file_writer + config.assets_dir = changed_assets + + decoded = Mock() + from_file = Mock(return_value=decoded) + monkeypatch.setattr(writer_module.AudioSegment, "from_file", from_file) + writer.add_audio_segment = Mock() + + writer.add_sound("tone") + + from_file.assert_called_once_with(sound_path) + writer.add_audio_segment.assert_called_once_with(decoded, None) diff --git a/tests/module/test_video_encoder.py b/tests/module/test_video_encoder.py new file mode 100644 index 0000000000..d3df2eed46 --- /dev/null +++ b/tests/module/test_video_encoder.py @@ -0,0 +1,217 @@ +from __future__ import annotations + +from dataclasses import replace +from fractions import Fraction + +import pytest + +from manim._config.output import OutputFormat, OutputSpec +from manim._config.video_encoder import ( + VideoEncoderSpec, + resolve_video_encoder, + to_av_frame_rate, + video_encoder_fingerprint, +) + + +def _output(format: OutputFormat, *, transparent: bool = False) -> OutputSpec: + return OutputSpec( + format=format, + transparent=transparent, + save_sections=False, + fallback_to_still=False, + ) + + +def _resolve( + format=OutputFormat.MP4, + *, + transparent=False, + width=1920, + height=1080, + frame_rate=30, + codec="auto", + pixel_format="auto", + options=None, +): + return resolve_video_encoder( + _output(format, transparent=transparent), + width=width, + height=height, + frame_rate=frame_rate, + codec=codec, + pixel_format=pixel_format, + options=options, + ) + + +@pytest.mark.parametrize( + ( + "format", + "transparent", + "container", + "codec", + "pixel_format", + "options", + ), + [ + (OutputFormat.MP4, False, "mp4", "libx264", "yuv420p", (("crf", "23"),)), + (OutputFormat.MOV, False, "mov", "libx264", "yuv420p", (("crf", "23"),)), + (OutputFormat.MOV, True, "mov", "qtrle", "argb", ()), + ( + OutputFormat.WEBM, + False, + "webm", + "libvpx-vp9", + "yuv420p", + (("crf", "23"),), + ), + ( + OutputFormat.WEBM, + True, + "webm", + "libvpx-vp9", + "yuva420p", + (("crf", "23"),), + ), + (OutputFormat.GIF, False, "mp4", "libx264", "yuv420p", (("crf", "23"),)), + (OutputFormat.GIF, True, "mov", "qtrle", "argb", ()), + ], +) +def test_default_video_encoder_profiles( + format, + transparent, + container, + codec, + pixel_format, + options, +): + spec = _resolve(format, transparent=transparent, frame_rate=60) + + assert spec == VideoEncoderSpec( + container_format=container, + codec=codec, + pixel_format=pixel_format, + width=1920, + height=1080, + frame_rate=Fraction(60, 1), + options=options, + ) + + +@pytest.mark.parametrize( + "format", + [OutputFormat.NONE, OutputFormat.PNG, OutputFormat.PNG_SEQUENCE], +) +def test_non_video_output_has_no_encoder(format): + assert _resolve(format, width=0, height=0, frame_rate=0) is None + + +@pytest.mark.parametrize( + ("value", "expected"), + [ + (25, Fraction(25, 1)), + (24.0, Fraction(24, 1)), + (23.976, Fraction(24_000, 1001)), + (23.98, Fraction(24_000, 1001)), + (59.94, Fraction(60_000, 1001)), + (12.5, Fraction(25, 2)), + (Fraction(30000, 1001), Fraction(30000, 1001)), + ], +) +def test_frame_rate_resolution(value, expected): + assert to_av_frame_rate(value) == expected + + +@pytest.mark.parametrize("value", [True, 0, -1, float("inf"), float("nan"), "60"]) +def test_invalid_frame_rates(value): + with pytest.raises(ValueError, match="frame rate"): + to_av_frame_rate(value) + + +@pytest.mark.parametrize( + ("width", "height"), [(0, 10), (10, -1), (1.5, 10), (True, 10)] +) +def test_invalid_video_geometry(width, height): + with pytest.raises(ValueError, match="dimensions"): + _resolve(width=width, height=height) + + +def test_custom_options_overlay_defaults_and_are_sorted(): + spec = _resolve( + codec="libx264", + options={"preset": "slow", "crf": "18"}, + ) + + assert spec is not None + assert spec.options == (("crf", "18"), ("preset", "slow")) + + +def test_different_explicit_codec_does_not_inherit_default_options(): + spec = _resolve(codec="qtrle", pixel_format="argb") + + assert spec is not None + assert spec.options == () + + +@pytest.mark.parametrize("option", ["codec", "pixel_format", "width", "rate"]) +def test_stream_fields_cannot_be_supplied_as_codec_options(option): + with pytest.raises(ValueError, match="explicit stream settings"): + _resolve(options={option: "value"}) + + +def test_encoder_options_must_be_strings(): + with pytest.raises(TypeError, match="keys and values must be strings"): + _resolve(options={"crf": 18}) + + +def test_unknown_encoder_is_rejected(): + with pytest.raises(ValueError, match="Unknown video encoder"): + _resolve(codec="not-a-codec") + + +def test_encoder_pixel_format_mismatch_is_rejected(): + with pytest.raises(ValueError, match="not supported"): + _resolve(codec="qtrle", pixel_format="yuv420p") + + +def test_video_encoder_fingerprint_is_canonical_and_byte_sensitive(): + spec = VideoEncoderSpec( + container_format="mp4", + codec="libx264", + pixel_format="yuv420p", + width=1920, + height=1080, + frame_rate=Fraction(30, 1), + options=(("preset", "slow"), ("crf", "18")), + ) + token = video_encoder_fingerprint(spec) + + assert len(token) == 16 + assert ( + video_encoder_fingerprint( + replace(spec, options=tuple(reversed(spec.options))), + ) + == token + ) + + changed_specs = [ + replace(spec, container_format="mov"), + replace(spec, codec="libx265"), + replace(spec, pixel_format="yuv444p"), + replace(spec, width=1280), + replace(spec, height=720), + replace(spec, frame_rate=Fraction(60, 1)), + replace(spec, options=(("crf", "19"), ("preset", "slow"))), + ] + assert all(video_encoder_fingerprint(changed) != token for changed in changed_specs) + assert video_encoder_fingerprint(None) == "none" + + +def test_transparent_output_requires_alpha_pixel_format(): + with pytest.raises(ValueError, match="alpha-bearing"): + _resolve( + OutputFormat.WEBM, + transparent=True, + pixel_format="yuv420p", + ) diff --git a/tests/module/test_video_segment_encoder.py b/tests/module/test_video_segment_encoder.py new file mode 100644 index 0000000000..86ec8f0439 --- /dev/null +++ b/tests/module/test_video_segment_encoder.py @@ -0,0 +1,159 @@ +from fractions import Fraction +from unittest.mock import Mock, call + +import av +import numpy as np +import pytest + +from manim._config.video_encoder import VideoEncoderSpec +from manim.scene.video_segment_encoder import VideoSegmentEncoder + + +def _spec(*, width=4, height=2): + return VideoEncoderSpec( + container_format="mp4", + codec="libx264", + pixel_format="yuv420p", + width=width, + height=height, + frame_rate=Fraction(30, 1), + options=(("crf", "23"),), + ) + + +def _detached_encoder(tmp_path, *, stream=None, container=None): + encoder = object.__new__(VideoSegmentEncoder) + encoder.target = tmp_path / "segment.mp4" + encoder.spec = _spec() + encoder._next_pts = 0 + encoder._closed = False + encoder._stream = Mock() if stream is None else stream + encoder._container = Mock() if container is None else container + return encoder + + +def _frame(): + return np.full((2, 4, 4), 127, dtype=np.uint8) + + +def test_write_frame_owns_repeat_and_segment_local_pts(tmp_path): + packet = object() + encoded_frames = [] + stream = Mock() + + def encode(frame): + encoded_frames.append(frame) + return [packet] + + stream.encode.side_effect = encode + container = Mock() + encoder = _detached_encoder(tmp_path, stream=stream, container=container) + + encoder.write_frame(_frame(), repeat=3) + + assert [frame.pts for frame in encoded_frames] == [0, 1, 2] + assert [frame.time_base for frame in encoded_frames] == [Fraction(1, 30)] * 3 + assert encoder._next_pts == 3 + assert container.mux.call_args_list == [call(packet)] * 3 + + +@pytest.mark.parametrize( + ("frame", "repeat", "exception", "message"), + [ + (np.zeros((3, 4, 4), dtype=np.uint8), 1, ValueError, "shape"), + (np.zeros((2, 4, 3), dtype=np.uint8), 1, ValueError, "shape"), + (np.zeros((2, 4, 4), dtype=np.float32), 1, TypeError, "uint8"), + (np.zeros((2, 4, 4), dtype=np.uint8)[:, ::-1], 1, ValueError, "C-contiguous"), + (np.zeros((2, 4, 4), dtype=np.uint8), 0, ValueError, "positive integer"), + (np.zeros((2, 4, 4), dtype=np.uint8), True, ValueError, "positive integer"), + ], +) +def test_write_frame_validates_boundary( + tmp_path, + frame, + repeat, + exception, + message, +): + encoder = _detached_encoder(tmp_path) + + with pytest.raises(exception, match=message): + encoder.write_frame(frame, repeat=repeat) + + encoder._stream.encode.assert_not_called() + + +def test_write_frame_rejects_closed_encoder(tmp_path): + encoder = _detached_encoder(tmp_path) + encoder._closed = True + + with pytest.raises(RuntimeError, match="is closed"): + encoder.write_frame(_frame()) + + +def test_encode_failure_has_target_profile_and_original_cause(tmp_path): + expected_exception = RuntimeError("codec exploded") + stream = Mock() + stream.encode.side_effect = expected_exception + encoder = _detached_encoder(tmp_path, stream=stream) + + with pytest.raises( + RuntimeError, match=r"segment\.mp4.*mp4/libx264/yuv420p" + ) as exc_info: + encoder.write_frame(_frame()) + + assert exc_info.value.__cause__ is expected_exception + + +def test_finish_flushes_closes_and_is_idempotent(tmp_path): + packet = object() + stream = Mock() + stream.encode.return_value = [packet] + container = Mock() + encoder = _detached_encoder(tmp_path, stream=stream, container=container) + + encoder.finish() + encoder.finish() + + stream.encode.assert_called_once_with() + container.mux.assert_called_once_with(packet) + container.close.assert_called_once_with() + + +def test_finish_preserves_flush_failure_when_close_also_fails(tmp_path): + flush_exception = RuntimeError("flush failed") + close_exception = RuntimeError("close failed") + stream = Mock() + stream.encode.side_effect = flush_exception + container = Mock() + container.close.side_effect = close_exception + encoder = _detached_encoder(tmp_path, stream=stream, container=container) + + with pytest.raises(RuntimeError, match="Failed to finish") as exc_info: + encoder.finish() + + assert exc_info.value.__cause__ is flush_exception + container.close.assert_called_once_with() + + +def test_abort_closes_removes_target_and_is_idempotent(tmp_path): + encoder = _detached_encoder(tmp_path) + encoder.target.write_bytes(b"incomplete") + + encoder.abort() + encoder.abort() + + encoder._container.close.assert_called_once_with() + assert not encoder.target.exists() + + +def test_open_failure_has_target_profile_and_original_cause(tmp_path, monkeypatch): + expected_exception = RuntimeError("open failed") + monkeypatch.setattr(av, "open", Mock(side_effect=expected_exception)) + + with pytest.raises( + RuntimeError, match=r"segment\.mp4.*mp4/libx264/yuv420p" + ) as exc_info: + VideoSegmentEncoder(target=tmp_path / "segment.mp4", spec=_spec()) + + assert exc_info.value.__cause__ is expected_exception diff --git a/tests/module/utils/test_caching.py b/tests/module/utils/test_caching.py new file mode 100644 index 0000000000..0356bad787 --- /dev/null +++ b/tests/module/utils/test_caching.py @@ -0,0 +1,166 @@ +from __future__ import annotations + +import os +from unittest.mock import Mock + +import numpy as np + +from manim.utils import caching +from manim.utils.caching import ( + clear_segment_cache, + handle_caching_play, + prune_segment_cache, +) + + +def test_prune_segment_cache_ignores_non_segments_and_hidden_files(tmp_path): + segment = tmp_path / "00001.mp4" + segment.touch() + ignored = [ + tmp_path / ".DS_Store", + tmp_path / "._00001.mp4", + tmp_path / "partial_movie_file_list.txt", + tmp_path / "notes.txt", + ] + for path in ignored: + path.touch() + + prune_segment_cache(tmp_path, 1) + assert segment.exists() + + prune_segment_cache(tmp_path, 0) + assert not segment.exists() + assert all(path.exists() for path in ignored) + + +def test_prune_segment_cache_removes_least_recently_accessed(tmp_path): + oldest = tmp_path / "oldest.mp4" + newest = tmp_path / "newest.mp4" + oldest.touch() + newest.touch() + os.utime(oldest, (1, 1)) + os.utime(newest, (2, 2)) + + prune_segment_cache(tmp_path, 1) + + assert not oldest.exists() + assert newest.exists() + + +def test_prune_segment_cache_treats_minus_one_as_unlimited(tmp_path): + segments = [tmp_path / f"{index}.mp4" for index in range(3)] + for segment in segments: + segment.touch() + + prune_segment_cache(tmp_path, -1) + + assert all(segment.exists() for segment in segments) + + +def test_prune_segment_cache_tolerates_vanishing_files(tmp_path, monkeypatch): + survivor = tmp_path / "00001.mp4" + survivor.touch() + ghost = tmp_path / "00002.mp4" + monkeypatch.setattr( + caching, + "_segment_cache_files", + lambda directory: [survivor, ghost], + ) + + prune_segment_cache(tmp_path, 0) + + assert not survivor.exists() + + +def test_prune_segment_cache_does_not_over_evict_for_vanished_file( + tmp_path, + monkeypatch, +): + survivors = [tmp_path / f"{index:05}.mp4" for index in range(2)] + for survivor in survivors: + survivor.touch() + ghost = tmp_path / "00002.mp4" + monkeypatch.setattr( + caching, + "_segment_cache_files", + lambda directory: [*survivors, ghost], + ) + + prune_segment_cache(tmp_path, len(survivors)) + + assert all(survivor.exists() for survivor in survivors) + + +def test_clear_segment_cache_removes_only_recognized_segments(tmp_path): + segments = [tmp_path / name for name in ("one.mp4", "two.mov", "three.webm")] + ignored = [ + tmp_path / ".hidden.mp4", + tmp_path / "partial_movie_file_list.txt", + tmp_path / "unrelated.mkv", + ] + for path in [*segments, *ignored]: + path.touch() + + assert clear_segment_cache(tmp_path) == 3 + assert all(not segment.exists() for segment in segments) + assert all(path.exists() for path in ignored) + + +def test_clear_segment_cache_accepts_missing_directory(tmp_path): + assert clear_segment_cache(tmp_path / "missing") == 0 + + +def test_opengl_cache_path_supplies_backend_encoder_and_raster_state(monkeypatch): + encoder = object() + fingerprint = Mock(return_value="encoder-token") + hash_play = Mock(return_value="cache-key") + monkeypatch.setattr(caching, "video_encoder_fingerprint", fingerprint) + monkeypatch.setattr(caching, "get_hash_from_play_call", hash_play) + + class FakeScene: + def __init__(self): + self.mobjects = [object()] + self.meshes = [object()] + self.session_spec = Mock(video_encoder=encoder) + + def compile_animations(self, *args, **kwargs): + return [] + + def add_mobjects_from_animations(self, animations): + pass + + class FakeRenderer: + _original_skipping_status = False + skip_animations = False + num_plays = 0 + animations_hashes = [] + camera = object() + background_color = np.array([0.1, 0.2, 0.3, 1.0]) + anti_alias_width = 1.5 + file_writer = Mock() + + def update_skipping_status(self): + pass + + @handle_caching_play + def play(self, scene, *args, **kwargs): + pass + + scene = FakeScene() + renderer = FakeRenderer() + renderer.file_writer.is_already_cached.return_value = False + + renderer.play(scene) + + fingerprint.assert_called_once_with(encoder) + hash_play.assert_called_once() + assert hash_play.call_args.kwargs == { + "backend": "opengl", + "encoder_fingerprint": "encoder-token", + "renderer_state": { + "meshes": scene.meshes, + "background_color": renderer.background_color, + "anti_alias_width": renderer.anti_alias_width, + }, + } + renderer.file_writer.add_partial_movie_file.assert_called_once_with("cache-key") diff --git a/tests/module/utils/test_file_ops.py b/tests/module/utils/test_file_ops.py index b56684eb0c..3cca3272f2 100644 --- a/tests/module/utils/test_file_ops.py +++ b/tests/module/utils/test_file_ops.py @@ -1,8 +1,10 @@ from __future__ import annotations from pathlib import Path +from unittest.mock import Mock, call from manim import * +from manim.utils import file_ops from tests.assert_utils import assert_dir_exists, assert_file_not_exists from tests.utils.video_tester import * @@ -29,3 +31,21 @@ def test_guarantee_empty_existence(tmp_path: Path): assert_dir_exists(test_dir) # test if dir got cleaned assert_file_not_exists(test_dir / "test.txt") + + +def test_open_media_file_can_reveal_and_preview(monkeypatch, tmp_path: Path): + artifact = tmp_path / "scene.mp4" + file_writer = Mock(final_file_path=artifact) + open_file = Mock() + monkeypatch.setattr(file_ops, "open_file", open_file) + + file_ops.open_media_file( + file_writer, + preview=True, + show_in_file_browser=True, + ) + + assert open_file.call_args_list == [ + call(artifact, True), + call(artifact, False), + ] diff --git a/tests/module/utils/test_hashing.py b/tests/module/utils/test_hashing.py index 788d86a787..3f63ff0530 100644 --- a/tests/module/utils/test_hashing.py +++ b/tests/module/utils/test_hashing.py @@ -11,6 +11,15 @@ from manim import ImageMobject, Square ALREADY_PROCESSED_PLACEHOLDER = hashing._Memoizer.ALREADY_PROCESSED_PLACEHOLDER +_CACHE_IDENTITY = { + "backend": "cairo", + "encoder_fingerprint": "encoder-token", + "renderer_state": (), +} + + +def _play_hash(*args): + return hashing.get_hash_from_play_call(*args, **_CACHE_IDENTITY) def test_JSON_basic(): @@ -254,13 +263,13 @@ def __str__(self) -> str: camera = HashableObject("camera", np.arange(8, dtype=np.uint8)) mobject = ImageMobject(np.zeros((8, 8, 4), dtype=np.uint8)) - original = hashing.get_hash_from_play_call(scene, camera, [], [mobject]) + original = _play_hash(scene, camera, [], [mobject]) mobject.pixel_array[4, 4, 0] ^= 1 - assert hashing.get_hash_from_play_call(scene, camera, [], [mobject]) != original + assert _play_hash(scene, camera, [], [mobject]) != original mobject.pixel_array[4, 4, 0] ^= 1 camera.pixel_array[0] ^= 1 - assert hashing.get_hash_from_play_call(scene, camera, [], [mobject]) == original + assert _play_hash(scene, camera, [], [mobject]) == original def test_play_hash_keeps_distinct_mobjects_with_equal_python_hashes(): @@ -278,9 +287,51 @@ def __hash__(self) -> int: camera = CollidingObject("camera") mobjects = [CollidingObject("first"), CollidingObject("second")] - original = hashing.get_hash_from_play_call(scene, camera, [], mobjects) + original = _play_hash(scene, camera, [], mobjects) mobjects[1].name = "changed" - assert hashing.get_hash_from_play_call(scene, camera, [], mobjects) != original + assert _play_hash(scene, camera, [], mobjects) != original + + +def test_play_hash_includes_backend_encoder_and_renderer_state(): + class HashableObject: + def __init__(self, value): + self.value = value + + scene = HashableObject("scene") + camera = HashableObject("camera") + + def cache_key(*, backend="cairo", encoder="encoder", **renderer_state): + return hashing.get_hash_from_play_call( + scene, + camera, + [], + [], + backend=backend, + encoder_fingerprint=encoder, + renderer_state=renderer_state, + ) + + renderer_state = { + "meshes": [HashableObject("mesh")], + "background_color": np.array([0.0, 0.0, 0.0, 1.0]), + "anti_alias_width": 1.5, + } + original = cache_key(**renderer_state) + changed_inputs = [ + cache_key(backend="opengl", **renderer_state), + cache_key(encoder="other", **renderer_state), + cache_key(**{**renderer_state, "meshes": [HashableObject("changed")]}), + cache_key( + **{ + **renderer_state, + "background_color": np.array([1.0, 0.0, 0.0, 1.0]), + }, + ), + cache_key(**{**renderer_state, "anti_alias_width": 2.0}), + ] + + assert len(original) == 64 + assert all(changed != original for changed in changed_inputs) def test_JSON_with_tuple(): diff --git a/tests/opengl/test_config_opengl.py b/tests/opengl/test_config_opengl.py index d0ca0e5a81..0250502ea6 100644 --- a/tests/opengl/test_config_opengl.py +++ b/tests/opengl/test_config_opengl.py @@ -6,6 +6,16 @@ import numpy as np from manim import WHITE, Scene, Square, tempconfig +from manim._config.render_session import resolve_render_session +from manim.renderer.protocol import RendererCapabilities + + +def _resolve_output(config): + return resolve_render_session( + config, + RendererCapabilities(live_preview=True), + renderer_name="TestRenderer", + ).output def test_tempconfig(config, using_opengl_renderer): @@ -112,15 +122,12 @@ def test_frame_size_if_frame_width(config, using_opengl_renderer, tmp_path): def test_temporary_dry_run(config, using_opengl_renderer): """Test that tempconfig correctly restores after setting dry_run.""" - assert config["write_to_movie"] - assert not config["save_last_frame"] + assert _resolve_output(config).is_video with tempconfig({"dry_run": True}): - assert not config["write_to_movie"] - assert not config["save_last_frame"] + assert not _resolve_output(config).enabled - assert config["write_to_movie"] - assert not config["save_last_frame"] + assert _resolve_output(config).is_video def test_dry_run_with_png_format(config, using_opengl_renderer, dry_run): @@ -135,7 +142,7 @@ def test_dry_run_with_png_format_skipped_animations( config, using_opengl_renderer, dry_run ): """Test that there are no exceptions when running a png without output and skipped animations""" - config.write_to_movie = False + config.format = "png" config.disable_caching = True assert config["dry_run"] is True scene = MyScene(skip_animations=True) diff --git a/tests/test_config.py b/tests/test_config.py index f80b5808ee..ffa1ea06cb 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -5,13 +5,30 @@ import numpy as np import pytest +from click.testing import CliRunner from manim import RIGHT, WHITE, Scene, Square, Tex, Text, Vector, tempconfig +from manim._config.output import OutputFormat, OutputSpec +from manim._config.render_session import resolve_render_session from manim._config.utils import ManimConfig +from manim.cli.render.commands import render from manim.constants import RendererType from manim.mobject.opengl.opengl_vectorized_mobject import OpenGLVMobject from manim.mobject.types.vectorized_mobject import VMobject -from tests.assert_utils import assert_dir_exists, assert_dir_filled, assert_file_exists +from manim.renderer.protocol import RendererCapabilities +from tests.assert_utils import assert_dir_filled, assert_file_exists + + +def _resolve_session(config): + return resolve_render_session( + config, + RendererCapabilities(live_preview=True), + renderer_name="TestRenderer", + ) + + +def _resolve_output(config): + return _resolve_session(config).output def test_tempconfig(config): @@ -37,6 +54,23 @@ def test_tempconfig(config): assert config[k] == v +def test_max_files_cached_uses_minus_one_for_unlimited(config): + config.max_files_cached = -1 + assert config.max_files_cached == -1 + + with pytest.raises(ValueError, match="non-negative integer or -1"): + config.max_files_cached = float("inf") + + +def test_custom_folders_config_option_was_removed(): + candidate = ManimConfig() + + assert not hasattr(ManimConfig, "custom_folders") + assert "custom_folders" not in candidate + with pytest.raises(AttributeError): + candidate["custom_folders"] = True + + def test_tempconfig_restores_renderer_class_bases(config): with tempconfig({"renderer": "opengl"}): assert config.renderer == RendererType.OPENGL @@ -57,9 +91,349 @@ def test_tempconfig_restores_renderer_class_bases(config): ("gif", ".mp4"), ], ) -def test_resolve_file_extensions(config, format, expected_file_extension): +def test_resolve_segment_extensions(config, format, expected_file_extension): + config.format = format + assert _resolve_output(config).segment_extension == expected_file_extension + + +@pytest.mark.parametrize( + ("format", "expected_format"), + [ + ("auto", OutputFormat.MP4), + ("none", OutputFormat.NONE), + ("png", OutputFormat.PNG), + ("png-sequence", OutputFormat.PNG_SEQUENCE), + ("gif", OutputFormat.GIF), + ], +) +def test_resolve_session_output(config, format, expected_format): config.format = format - assert config.movie_file_extension == expected_file_extension + + assert _resolve_output(config).format is expected_format + + +def test_transparent_auto_output_resolves_to_mov(config): + config.format = "auto" + config.transparent = True + + output = _resolve_output(config) + + assert output.format is OutputFormat.MOV + assert output.fallback_to_still is True + + +def test_only_automatic_video_output_allows_still_fallback(config): + config.format = "auto" + assert _resolve_output(config).fallback_to_still is True + + config.format = "mp4" + assert _resolve_output(config).fallback_to_still is False + + +def test_still_fallback_requires_video_output(): + with pytest.raises(ValueError, match="requires a video output format"): + OutputSpec( + OutputFormat.PNG, + transparent=False, + save_sections=False, + fallback_to_still=True, + ) + + +def test_explicit_no_output_is_not_dry_run(config): + config.format = "none" + + session = _resolve_session(config) + + assert session.output.format is OutputFormat.NONE + assert session.dry_run is False + + +def test_live_preview_auto_output_resolves_to_none(config): + config.format = "auto" + config.live_preview = True + + session = _resolve_session(config) + + assert session.output.format is OutputFormat.NONE + assert session.output.fallback_to_still is False + assert session.presentation.live_preview is True + assert session.dry_run is False + + +def test_live_preview_requires_renderer_capability(config): + config.live_preview = True + + with pytest.raises(ValueError, match="does not support live preview"): + resolve_render_session( + config, + RendererCapabilities(), + renderer_name="TestRenderer", + ) + + +def test_preview_requires_output(config): + config.format = "none" + config.preview = True + + with pytest.raises(ValueError, match="requires a media artifact"): + resolve_render_session( + config, + RendererCapabilities(), + renderer_name="TestRenderer", + ) + + +def test_explicit_transparent_mp4_is_rejected(config): + config.format = "mp4" + config.transparent = True + + with pytest.raises(ValueError, match="does not support an alpha channel"): + _resolve_output(config) + + +def test_dry_run_resolves_no_output_without_mutating_output_request(config): + config.format = "gif" + config.save_sections = True + config.dry_run = True + + session = _resolve_session(config) + + assert session.output.format is OutputFormat.NONE + assert session.output.save_sections is False + assert session.output.fallback_to_still is False + assert session.dry_run is True + assert config.format == "gif" + assert config.save_sections is True + + +def test_save_last_frame_resolves_to_still_output(config): + config.format = "auto" + config.save_last_frame = True + + assert _resolve_output(config).format is OutputFormat.PNG + + +def test_save_last_frame_alias_works_with_tempconfig(config): + original_format = config.format + + with tempconfig({"save_last_frame": True}): + assert config.format == "png" + assert _resolve_output(config).is_still + + assert config.format == original_format + + +def test_sections_require_video_output(config): + config.format = "png" + config.save_sections = True + + with pytest.raises(ValueError, match="Section output requires"): + _resolve_output(config) + + +def test_format_is_loaded_from_config_file(tmp_path, config): + config_file = tmp_path / "output.cfg" + config_file.write_text("[CLI]\nformat = png-sequence\n") + + config.digest_file(config_file) + + assert config.format == "png-sequence" + + +def test_render_session_resolves_configured_encoder_profile(config): + config.format = "mov" + config.video_codec = "qtrle" + config.pixel_format = "argb" + config.video_encoder_options = {"threads": "1"} + + encoder = _resolve_session(config).video_encoder + + assert encoder is not None + assert encoder.codec == "qtrle" + assert encoder.pixel_format == "argb" + assert encoder.options == (("threads", "1"),) + + +def test_video_encoder_profile_is_loaded_from_config_file(tmp_path, config): + config_file = tmp_path / "encoder.cfg" + config_file.write_text( + """\ +[video_encoder] +codec = libx264 +pixel_format = yuv444p + +[video_encoder.options] +crf = 18 +preset = veryslow +tune = animation, film: grain +""", + ) + + config.digest_file(config_file) + + assert config.video_codec == "libx264" + assert config.pixel_format == "yuv444p" + assert config.video_encoder_options == { + "crf": "18", + "preset": "veryslow", + "tune": "animation, film: grain", + } + + +def test_media_loglevel_is_loaded_from_media_section(tmp_path, config): + config_file = tmp_path / "media.cfg" + config_file.write_text("[media]\nloglevel = DEBUG\n") + + config.digest_file(config_file) + + assert config.media_loglevel == "DEBUG" + with pytest.raises(AttributeError): + _ = config.ffmpeg_loglevel + + +def test_video_encoder_options_are_copied(config): + supplied = {"crf": "18"} + config.video_encoder_options = supplied + supplied["crf"] = "30" + assert config.video_encoder_options == {"crf": "18"} + + returned = config.video_encoder_options + returned["crf"] = "40" + assert config.video_encoder_options == {"crf": "18"} + + +def test_encoder_cli_options_replace_config_file_map(tmp_path, config): + scene_file = tmp_path / "trivial_scene.py" + scene_file.write_text("# --jupyter returns before importing this file\n") + config_file = tmp_path / "encoder.cfg" + config_file.write_text( + """\ +[video_encoder] +codec = qtrle +pixel_format = argb + +[video_encoder.options] +predictor = 1 +""", + ) + runner = CliRunner() + common_args = [ + str(scene_file), + "--jupyter", + "--config_file", + str(config_file), + ] + + result = runner.invoke( + render, + [ + *common_args, + "--video-codec", + "libx264", + "--pixel-format", + "yuv420p", + "--encoder-option", + "crf=18", + "--encoder-option", + "tune=animation=film", + ], + standalone_mode=False, + ) + assert result.exit_code == 0, result.output + with tempconfig({}): + config.digest_args(result.return_value) + assert config.video_codec == "libx264" + assert config.pixel_format == "yuv420p" + assert config.video_encoder_options == { + "crf": "18", + "tune": "animation=film", + } + + result = runner.invoke(render, common_args, standalone_mode=False) + assert result.exit_code == 0, result.output + with tempconfig({}): + config.digest_args(result.return_value) + assert config.video_codec == "qtrle" + assert config.pixel_format == "argb" + assert config.video_encoder_options == {"predictor": "1"} + + +@pytest.mark.parametrize( + "options", + [ + ["--encoder-option", "missing-separator"], + ["--encoder-option", "=missing-key"], + ["--encoder-option", "missing-value="], + ["--encoder-option", "crf=18", "--encoder-option", "crf=20"], + ], +) +def test_encoder_cli_options_reject_malformed_or_duplicate_entries(tmp_path, options): + scene_file = tmp_path / "trivial_scene.py" + scene_file.touch() + + result = CliRunner().invoke(render, [str(scene_file), *options]) + + assert result.exit_code == 2 + assert "Invalid value" in result.output + + +def test_cli_distinguishes_preview_from_live_preview(tmp_path): + scene_file = tmp_path / "scene.py" + scene_file.write_text("# --jupyter returns before loading this file\n") + + result = CliRunner().invoke( + render, + [str(scene_file), "--jupyter", "--preview", "--live-preview"], + standalone_mode=False, + ) + + assert result.exception is None + assert result.return_value.preview is True + assert result.return_value.live_preview is True + + +def test_opengl_cli_no_longer_disables_automatic_output(tmp_path, config): + scene_file = tmp_path / "scene.py" + scene_file.write_text("# --jupyter returns before loading this file\n") + result = CliRunner().invoke( + render, + [str(scene_file), "--jupyter", "--renderer=opengl"], + standalone_mode=False, + ) + assert result.exception is None + + config.format = "auto" + config.digest_args(result.return_value) + + assert config.renderer is RendererType.OPENGL + assert config.format == "auto" + assert _resolve_output(config).format is OutputFormat.MP4 + + +def test_absent_cli_output_options_preserve_config_file_values(tmp_path): + scene_file = tmp_path / "scene.py" + scene_file.write_text("# --jupyter returns before loading this file\n") + config_file = tmp_path / "output.cfg" + config_file.write_text( + "[CLI]\n" + "format = webm\n" + "output_file = configured-name\n" + "background_opacity = 0.5\n", + ) + result = CliRunner().invoke( + render, + [str(scene_file), "--jupyter"], + standalone_mode=False, + ) + assert result.exception is None + + candidate = ManimConfig().digest_file(config_file) + candidate.digest_args(result.return_value) + + assert candidate.format == "webm" + assert candidate.output_file == "configured-name" + assert candidate.transparent is True class MyScene(Scene): @@ -96,7 +470,6 @@ def test_transparent_by_background_opacity(config, dry_run): scene.render() frame = scene.renderer.get_frame() np.testing.assert_allclose(frame[0, 0], [0, 0, 0, 127]) - assert config.movie_file_extension == ".mov" assert config.transparent is True @@ -160,8 +533,7 @@ def test_custom_dirs(tmp_path, config): assert_dir_filled(tmp_path / "test_partial_movie_dir") assert_file_exists(tmp_path / "test_partial_movie_dir/partial_movie_file_list.txt") - # TODO: another example with image output would be nice - assert_dir_exists(tmp_path / "test_images") + assert not (tmp_path / "test_images").exists() assert_dir_filled(tmp_path / "test_text") assert_dir_filled(tmp_path / "test_tex") @@ -213,20 +585,17 @@ def test_frame_size(tmp_path, config): def test_temporary_dry_run(config): """Test that tempconfig correctly restores after setting dry_run.""" - assert config["write_to_movie"] - assert not config["save_last_frame"] + assert _resolve_output(config).is_video with tempconfig({"dry_run": True}): - assert not config["write_to_movie"] - assert not config["save_last_frame"] + assert not _resolve_output(config).enabled - assert config["write_to_movie"] - assert not config["save_last_frame"] + assert _resolve_output(config).is_video def test_dry_run_with_png_format(config, dry_run): """Test that there are no exceptions when running a png without output""" - config.write_to_movie = False + config.format = "png" config.disable_caching = True assert config.dry_run is True scene = MyScene() @@ -235,7 +604,7 @@ def test_dry_run_with_png_format(config, dry_run): def test_dry_run_with_png_format_skipped_animations(config, dry_run): """Test that there are no exceptions when running a png without output and skipped animations""" - config.write_to_movie = False + config.format = "png" config.disable_caching = True assert config["dry_run"] is True scene = MyScene(skip_animations=True) diff --git a/tests/test_logging/test_logging.py b/tests/test_logging/test_logging.py index 397573a51e..edf13aafe2 100644 --- a/tests/test_logging/test_logging.py +++ b/tests/test_logging/test_logging.py @@ -30,6 +30,29 @@ def test_logging_to_file(tmp_path, python_version): assert exitcode == 0, err +def test_library_logging_without_media_output(tmp_path, python_version): + script = f""" +from manim import Scene, tempconfig + +class LibraryScene(Scene): + pass + +with tempconfig({{ + "format": "none", + "log_to_file": True, + "media_dir": {str(tmp_path)!r}, +}}): + LibraryScene().render() +""" + + _, err, exitcode = capture([python_version, "-c", script]) + + assert exitcode == 0, err + assert (tmp_path / "logs" / "_LibraryScene.log").is_file() + assert not (tmp_path / "videos").exists() + assert not (tmp_path / "images").exists() + + def test_error_logging(tmp_path, python_version): path_error_scene = Path("tests/test_logging/basic_scenes_error.py") diff --git a/tests/test_scene_rendering/conftest.py b/tests/test_scene_rendering/conftest.py index 7263a3f37c..848e959caf 100644 --- a/tests/test_scene_rendering/conftest.py +++ b/tests/test_scene_rendering/conftest.py @@ -43,12 +43,12 @@ def infallible_scenes_path(): @pytest.fixture -def force_window_config_write_to_movie(config): - config.force_window = True - config.write_to_movie = True +def live_preview_config_movie(config): + config.live_preview = True + config.format = "mp4" @pytest.fixture -def force_window_config_pngs(config): - config.force_window = True - config.format = "png" +def live_preview_config_pngs(config): + config.live_preview = True + config.format = "png-sequence" diff --git a/tests/test_scene_rendering/opengl/test_caching_related_opengl.py b/tests/test_scene_rendering/opengl/test_caching_related_opengl.py index c9c82a449b..142ece5e24 100644 --- a/tests/test_scene_rendering/opengl/test_caching_related_opengl.py +++ b/tests/test_scene_rendering/opengl/test_caching_related_opengl.py @@ -23,7 +23,7 @@ def test_wait_skip(tmp_path, manim_cfg_file, simple_scenes_path): "manim", "--renderer", "opengl", - "--write_to_movie", + "--format=mp4", "-ql", "--media_dir", str(tmp_path), @@ -50,7 +50,7 @@ def test_play_skip(tmp_path, manim_cfg_file, simple_scenes_path): "manim", "--renderer", "opengl", - "--write_to_movie", + "--format=mp4", "-ql", "--media_dir", str(tmp_path), diff --git a/tests/test_scene_rendering/opengl/test_cli_flags_opengl.py b/tests/test_scene_rendering/opengl/test_cli_flags_opengl.py index 17f6f35e38..ac8079162f 100644 --- a/tests/test_scene_rendering/opengl/test_cli_flags_opengl.py +++ b/tests/test_scene_rendering/opengl/test_cli_flags_opengl.py @@ -26,7 +26,7 @@ def test_basic_scene_with_default_values(tmp_path, manim_cfg_file, simple_scenes "manim", "--renderer", "opengl", - "--write_to_movie", + "--format=mp4", "--media_dir", str(tmp_path), str(simple_scenes_path), @@ -87,7 +87,7 @@ def test_basic_scene_l_flag(tmp_path, manim_cfg_file, simple_scenes_path): "--renderer", "opengl", "-ql", - "--write_to_movie", + "--format=mp4", "--media_dir", str(tmp_path), str(simple_scenes_path), @@ -111,7 +111,7 @@ def test_n_flag(tmp_path, simple_scenes_path): "-ql", "--renderer", "opengl", - "--write_to_movie", + "--format=mp4", "-n 3,6", "--media_dir", str(tmp_path), @@ -166,11 +166,19 @@ def test_image_output_for_static_scene(tmp_path, manim_cfg_file, simple_scenes_p out, err, exit_code = capture(command) assert exit_code == 0, err - exists = (tmp_path / "videos").exists() - assert not exists, "running manim with static scene rendered a video" + unexpected_movie_path = ( + tmp_path / "videos" / "simple_scenes" / "480p15" / "StaticScene.mp4" + ) + assert not unexpected_movie_path.exists(), ( + "running manim with a static scene rendered a video" + ) - is_empty = not any((tmp_path / "images" / "simple_scenes").iterdir()) - assert not is_empty, "running manim without animations did not render an image" + expected_image_path = add_version_before_extension( + tmp_path / "images" / "simple_scenes" / "StaticScene.png", + ) + assert expected_image_path.exists(), ( + "running manim without animations did not render an image" + ) @pytest.mark.slow @@ -194,17 +202,20 @@ def test_no_image_output_with_interactive_embed( out, err, exit_code = capture(command) assert exit_code == 0, err - exists = (tmp_path / "videos").exists() - assert not exists, "running manim with static scene rendered a video" + unexpected_movie_path = ( + tmp_path / "videos" / "simple_scenes" / "480p15" / "InteractiveStaticScene.mp4" + ) + assert not unexpected_movie_path.exists(), ( + "running an interactive static scene rendered a video" + ) - is_empty = not any((tmp_path / "images" / "simple_scenes").iterdir()) - assert is_empty, ( - "running manim static scene with interactive embed rendered an image" + assert not (tmp_path / "images").exists(), ( + "running an interactive static scene rendered an image" ) @pytest.mark.slow -def test_no_default_image_output_with_non_static_scene( +def test_default_video_output_with_non_static_scene( tmp_path, manim_cfg_file, simple_scenes_path ): scene_name = "SceneWithNonStaticWait" @@ -223,25 +234,26 @@ def test_no_default_image_output_with_non_static_scene( out, err, exit_code = capture(command) assert exit_code == 0, err - exists = (tmp_path / "videos").exists() - assert not exists, "running manim with static scene rendered a video" + expected_movie_path = ( + tmp_path / "videos" / "simple_scenes" / "480p15" / "SceneWithNonStaticWait.mp4" + ) + assert expected_movie_path.exists(), ( + "default output did not render the non-static scene as a video" + ) - is_empty = not any((tmp_path / "images" / "simple_scenes").iterdir()) - assert is_empty, ( - "running manim static scene with interactive embed rendered an image" + assert not (tmp_path / "images").exists(), ( + "default video output unexpectedly rendered an image" ) @pytest.mark.slow -def test_image_output_for_static_scene_with_write_to_movie( - tmp_path, manim_cfg_file, simple_scenes_path -): +def test_explicit_video_output_for_static_scene_fails(tmp_path, simple_scenes_path): scene_name = "StaticScene" command = [ sys.executable, "-m", "manim", - "--write_to_movie", + "--format=mp4", "--renderer", "opengl", "-ql", @@ -250,14 +262,12 @@ def test_image_output_for_static_scene_with_write_to_movie( str(simple_scenes_path), scene_name, ] - out, err, exit_code = capture(command) - assert exit_code == 0, err - - is_empty = not any((tmp_path / "videos").iterdir()) - assert not is_empty, "running manim with static scene rendered a video" + _, err, exit_code = capture(command) - is_empty = not any((tmp_path / "images" / "simple_scenes").iterdir()) - assert not is_empty, "running manim without animations did not render an image" + assert exit_code == 1 + assert "explicitly requested MP4" in err + assert not list(tmp_path.rglob("*.mp4")) + assert not list(tmp_path.rglob("*.png")) @pytest.mark.slow @@ -324,7 +334,6 @@ def test_a_flag(tmp_path, manim_cfg_file, infallible_scenes_path): "manim", "--renderer", "opengl", - "--write_to_movie", "-ql", "--media_dir", str(tmp_path), @@ -354,33 +363,6 @@ def test_a_flag(tmp_path, manim_cfg_file, infallible_scenes_path): ) -@pytest.mark.slow -def test_custom_folders(tmp_path, manim_cfg_file, simple_scenes_path): - scene_name = "SquareToCircle" - command = [ - sys.executable, - "-m", - "manim", - "--renderer", - "opengl", - "-ql", - "-s", - "--media_dir", - str(tmp_path), - "--custom_folders", - str(simple_scenes_path), - scene_name, - ] - out, err, exit_code = capture(command) - assert exit_code == 0, err - - exists = (tmp_path / "videos").exists() - assert not exists, "--custom_folders produced a 'videos/' dir" - - exists = add_version_before_extension(tmp_path / "SquareToCircle.png").exists() - assert exists, "--custom_folders did not produce the output file" - - @pytest.mark.slow def test_dash_as_filename(tmp_path): code = ( @@ -518,12 +500,12 @@ def test_videos_not_created_when_png_format_set( @pytest.mark.slow -def test_images_are_created_when_png_format_set( +def test_images_are_created_when_png_sequence_format_set( tmp_path, manim_cfg_file, simple_scenes_path, ): - """Test images are created in media directory when --format png is set""" + """Test OpenGL images are created when --format png-sequence is set.""" scene_name = "SquareToCircle" command = [ sys.executable, @@ -535,24 +517,26 @@ def test_images_are_created_when_png_format_set( "--media_dir", str(tmp_path), "--format", - "png", + "png-sequence", str(simple_scenes_path), scene_name, ] out, err, exit_code = capture(command) assert exit_code == 0, err - expected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle0000.png" + expected_png_path = ( + tmp_path / "images" / "simple_scenes" / "SquareToCircle" / "0000.png" + ) assert expected_png_path.exists(), "png file not found at " + str(expected_png_path) @pytest.mark.slow -def test_images_are_zero_padded_when_zero_pad_set( +def test_png_sequence_images_are_zero_padded( tmp_path, manim_cfg_file, simple_scenes_path, ): - """Test images are zero padded when --format png and --zero_pad n are set""" + """Test OpenGL PNG-sequence images respect --zero_pad.""" scene_name = "SquareToCircle" command = [ sys.executable, @@ -564,7 +548,7 @@ def test_images_are_zero_padded_when_zero_pad_set( "--media_dir", str(tmp_path), "--format", - "png", + "png-sequence", "--zero_pad", "3", str(simple_scenes_path), @@ -573,12 +557,13 @@ def test_images_are_zero_padded_when_zero_pad_set( out, err, exit_code = capture(command) assert exit_code == 0, err - unexpected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle0.png" + sequence_dir = tmp_path / "images" / "simple_scenes" / "SquareToCircle" + unexpected_png_path = sequence_dir / "0.png" assert not unexpected_png_path.exists(), "non zero padded png file found at " + str( unexpected_png_path, ) - expected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle000.png" + expected_png_path = sequence_dir / "000.png" assert expected_png_path.exists(), "png file not found at " + str(expected_png_path) @@ -633,7 +618,7 @@ def test_default_format_output_for_transparent_flag( "--renderer", "opengl", "-ql", - "--write_to_movie", + "--format=auto", "--media_dir", str(tmp_path), "-t", diff --git a/tests/test_scene_rendering/opengl/test_opengl_renderer.py b/tests/test_scene_rendering/opengl/test_opengl_renderer.py index f2236cb04d..c2925b7372 100644 --- a/tests/test_scene_rendering/opengl/test_opengl_renderer.py +++ b/tests/test_scene_rendering/opengl/test_opengl_renderer.py @@ -11,44 +11,44 @@ from tests.test_scene_rendering.simple_scenes import * -def test_write_to_movie_disables_window( +def test_file_output_disables_window( config, using_temp_opengl_config, disabling_caching ): - """write_to_movie should disable window by default""" + """File output should disable the window by default.""" scene = SquareToCircle() renderer = scene.renderer renderer.update_frame = Mock(wraps=renderer.update_frame) scene.render() assert renderer.window is None - assert_file_exists(config.output_file) + assert_file_exists(renderer.file_writer.final_file_path) @pytest.mark.skip(reason="Temporarily skip due to failing in Windows CI") -def test_force_window_opengl_render_with_movies( +def test_live_preview_opengl_render_with_movies( config, using_temp_opengl_config, - force_window_config_write_to_movie, + live_preview_config_movie, disabling_caching, ): - """force_window creates window when write_to_movie is set""" + """Live preview can be displayed while movie output is enabled.""" scene = SquareToCircle() renderer = scene.renderer renderer.update_frame = Mock(wraps=renderer.update_frame) scene.render() assert renderer.window is not None - assert_file_exists(config["output_file"]) + assert_file_exists(renderer.file_writer.final_file_path) renderer.window.close() @pytest.mark.skipif( platform.processor() == "aarch64", reason="Fails on Linux-ARM runners" ) -def test_force_window_opengl_render_with_format( +def test_live_preview_opengl_render_with_image_sequence( using_temp_opengl_config, - force_window_config_pngs, + live_preview_config_pngs, disabling_caching, ): - """force_window creates window when format is set""" + """Live preview can be displayed while an image sequence is written.""" scene = SquareToCircle() renderer = scene.renderer renderer.update_frame = Mock(wraps=renderer.update_frame) @@ -57,13 +57,13 @@ def test_force_window_opengl_render_with_format( renderer.window.close() -def test_get_frame_with_preview_disabled(config, using_opengl_renderer): - """Get frame is able to fetch frame with the correct dimensions when preview is disabled""" - config.preview = False +def test_get_frame_with_live_preview_disabled(config, using_opengl_renderer): + """Get frame has the correct dimensions without a live preview.""" + config.live_preview = False scene = SquareToCircle() assert isinstance(scene.renderer, OpenGLRenderer) - assert not config.preview + assert not config.live_preview renderer = scene.renderer renderer.update_frame(scene) @@ -72,28 +72,50 @@ def test_get_frame_with_preview_disabled(config, using_opengl_renderer): # height and width are flipped assert renderer.get_pixel_shape()[0] == frame.shape[1] assert renderer.get_pixel_shape()[1] == frame.shape[0] + assert frame.dtype == np.uint8 + assert frame.flags.c_contiguous @pytest.mark.slow -def test_get_frame_with_preview_enabled(config, using_opengl_renderer): - """Get frame is able to fetch frame with the correct dimensions when preview is enabled""" - config.preview = True +def test_get_frame_with_live_preview_enabled(config, using_opengl_renderer): + """Get frame has the correct dimensions with a live preview.""" + config.live_preview = True scene = SquareToCircle() assert isinstance(scene.renderer, OpenGLRenderer) - assert config.preview is True + assert config.live_preview is True renderer = scene.renderer + assert renderer.window is not None + assert not renderer.file_writer.output_spec.enabled renderer.update_frame(scene) frame = renderer.get_frame() # height and width are flipped assert renderer.get_pixel_shape()[0] == frame.shape[1] assert renderer.get_pixel_shape()[1] == frame.shape[0] + assert frame.dtype == np.uint8 + assert frame.flags.c_contiguous + renderer.window.close() + + +def test_render_without_frame_output_skips_gpu_readback( + config, + using_opengl_renderer, +): + config.format = "none" + config.live_preview = False + scene = SquareToCircle() + renderer = scene.renderer + renderer.get_frame = Mock(wraps=renderer.get_frame) + + renderer.render(scene, 0, []) + + renderer.get_frame.assert_not_called() def test_pixel_coords_to_space_coords(config, using_opengl_renderer): - config.preview = True + config.live_preview = False scene = SquareToCircle() assert isinstance(scene.renderer, OpenGLRenderer) diff --git a/tests/test_scene_rendering/opengl/test_play_logic_opengl.py b/tests/test_scene_rendering/opengl/test_play_logic_opengl.py index 64c4c39204..9a49ef9ffb 100644 --- a/tests/test_scene_rendering/opengl/test_play_logic_opengl.py +++ b/tests/test_scene_rendering/opengl/test_play_logic_opengl.py @@ -93,10 +93,13 @@ def test_t_values_with_cached_data(using_temp_opengl_config): assert scene.update_to_time.call_count == 10 -@pytest.mark.xfail(reason="Not currently handled correctly for opengl") -def test_t_values_save_last_frame(config, using_temp_opengl_config): - """Test that there is only one t value handled when only saving the last frame""" - config.save_last_frame = True +@pytest.mark.parametrize( + "still_config", + [{"save_last_frame": True}, {"format": "png"}], +) +def test_t_values_save_last_frame(config, using_temp_opengl_config, still_config): + """Still output fast-forwards each play and only evaluates its final state.""" + config.update(still_config) scene = SquareToCircle() scene.update_to_time = Mock() scene.render() diff --git a/tests/test_scene_rendering/test_cairo_renderer.py b/tests/test_scene_rendering/test_cairo_renderer.py index 5134578aa3..0a2f38d28f 100644 --- a/tests/test_scene_rendering/test_cairo_renderer.py +++ b/tests/test_scene_rendering/test_cairo_renderer.py @@ -18,7 +18,28 @@ def test_render(using_temp_config, disabling_caching): scene.render() assert renderer.add_frame.call_count == config["frame_rate"] assert renderer.update_frame.call_count == config["frame_rate"] - assert_file_exists(config["output_file"]) + assert_file_exists(renderer.file_writer.final_file_path) + assert config.output_file == "" + + +def test_automatic_output_uses_still_for_static_scene(using_temp_config): + scene = StaticScene() + + scene.render() + + assert scene.renderer.file_writer.output_spec.fallback_to_still is True + assert scene.renderer.file_writer.final_file_path.suffix == ".png" + assert_file_exists(scene.renderer.file_writer.final_file_path) + + +def test_explicit_video_output_for_static_scene_fails(using_temp_config): + config.format = "mp4" + scene = StaticScene() + + with pytest.raises(RuntimeError, match="explicitly requested MP4"): + scene.render() + + assert not hasattr(scene.renderer.file_writer, "final_file_path") def test_skipping_status_with_from_to_and_up_to(using_temp_config, disabling_caching): @@ -60,7 +81,7 @@ def test_when_animation_is_cached(using_temp_config): # Check that manim correctly skipped the animation. scene.update_to_time.assert_called_once_with(1) # Check that the output video has been generated. - assert_file_exists(config["output_file"]) + assert_file_exists(scene.renderer.file_writer.final_file_path) def test_hash_logic_is_not_called_when_caching_is_disabled( @@ -71,7 +92,7 @@ def test_hash_logic_is_not_called_when_caching_is_disabled( scene = SquareToCircle() scene.render() mocked.assert_not_called() - assert_file_exists(config["output_file"]) + assert_file_exists(scene.renderer.file_writer.final_file_path) def test_hash_logic_is_called_when_caching_is_enabled(using_temp_config): @@ -84,3 +105,5 @@ def test_hash_logic_is_called_when_caching_is_enabled(using_temp_config): scene = SquareToCircle() scene.render() mocked.assert_called_once() + assert mocked.call_args.kwargs["backend"] == "cairo" + assert mocked.call_args.kwargs["encoder_fingerprint"] != "none" diff --git a/tests/test_scene_rendering/test_cli_flags.py b/tests/test_scene_rendering/test_cli_flags.py index 282dd1b50a..5b8348a355 100644 --- a/tests/test_scene_rendering/test_cli_flags.py +++ b/tests/test_scene_rendering/test_cli_flags.py @@ -140,14 +140,15 @@ def test_s_flag_no_animations(tmp_path, manim_cfg_file, simple_scenes_path): @pytest.mark.slow -def test_s_flag(tmp_path, manim_cfg_file, simple_scenes_path): +@pytest.mark.parametrize("still_flag", ["-s", "--format=png"]) +def test_s_flag(tmp_path, manim_cfg_file, simple_scenes_path, still_flag): scene_name = "SquareToCircle" command = [ sys.executable, "-m", "manim", "-ql", - "-s", + still_flag, "--media_dir", str(tmp_path), str(simple_scenes_path), @@ -252,29 +253,51 @@ def test_a_flag(tmp_path, manim_cfg_file, infallible_scenes_path): ) -@pytest.mark.slow -def test_custom_folders(tmp_path, manim_cfg_file, simple_scenes_path): - scene_name = "SquareToCircle" +@pytest.mark.parametrize( + "scene_selection", + [ + ("-a",), + ("Wait1", "Wait3"), + ], +) +def test_output_file_rejects_multi_scene_render( + tmp_path, + infallible_scenes_path, + scene_selection, +): command = [ sys.executable, "-m", "manim", - "-ql", - "-s", "--media_dir", str(tmp_path), - "--custom_folders", - str(simple_scenes_path), - scene_name, + "-o", + "shared-name", ] + if scene_selection == ("-a",): + command.extend(["-a", str(infallible_scenes_path)]) + else: + command.extend([str(infallible_scenes_path), *scene_selection]) + out, err, exit_code = capture(command) - assert exit_code == 0, err - exists = (tmp_path / "videos").exists() - assert not exists, "--custom_folders produced a 'videos/' dir" + assert exit_code == 1 + assert "--output_file can only be used when rendering exactly one scene" in ( + err or out + ) + assert not tmp_path.exists() or not any(tmp_path.iterdir()) - exists = add_version_before_extension(tmp_path / "SquareToCircle.png").exists() - assert exists, "--custom_folders did not produce the output file" + +def test_custom_folders_option_was_removed(simple_scenes_path): + runner = CliRunner() + + result = runner.invoke( + main, + ["--custom_folders", str(simple_scenes_path), "SquareToCircle"], + ) + + assert result.exit_code == 2 + assert "No such option: --custom_folders" in result.output @pytest.mark.slow @@ -323,7 +346,8 @@ def test_custom_output_name_gif(tmp_path, simple_scenes_path): @pytest.mark.slow def test_custom_output_name_mp4(tmp_path, simple_scenes_path): scene_name = "SquareToCircle" - custom_name = "custom_name" + requested_name = "custom_name.mov" + expected_name = requested_name command = [ sys.executable, "-m", @@ -332,7 +356,7 @@ def test_custom_output_name_mp4(tmp_path, simple_scenes_path): "--media_dir", str(tmp_path), "-o", - custom_name, + requested_name, str(simple_scenes_path), scene_name, ] @@ -344,18 +368,18 @@ def test_custom_output_name_mp4(tmp_path, simple_scenes_path): ) assert not wrong_mp4_path.exists(), ( - "The mp4 file does not respect the custom name: " + custom_name + ".mp4" + "The mp4 file does not respect the custom name: " + expected_name + ".mp4" ) unexpected_gif_path = add_version_before_extension( - tmp_path / "videos" / "simple_scenes" / "480p15" / f"{custom_name}.gif" + tmp_path / "videos" / "simple_scenes" / "480p15" / f"{expected_name}.gif" ) assert not unexpected_gif_path.exists(), "Found an unexpected gif file at " + str( unexpected_gif_path ) expected_mp4_path = ( - tmp_path / "videos" / "simple_scenes" / "480p15" / str(custom_name + ".mp4") + tmp_path / "videos" / "simple_scenes" / "480p15" / str(expected_name + ".mp4") ) assert expected_mp4_path.exists(), "mp4 file not found at " + str(expected_mp4_path) @@ -490,12 +514,12 @@ def test_videos_not_created_when_png_format_set( @pytest.mark.slow -def test_images_are_created_when_png_format_set( +def test_images_are_created_when_png_sequence_format_set( tmp_path, manim_cfg_file, simple_scenes_path, ): - """Test images are created in media directory when --format png is set""" + """Test images are created when --format png-sequence is set.""" scene_name = "SquareToCircle" command = [ sys.executable, @@ -505,24 +529,26 @@ def test_images_are_created_when_png_format_set( "--media_dir", str(tmp_path), "--format", - "png", + "png-sequence", str(simple_scenes_path), scene_name, ] out, err, exit_code = capture(command) assert exit_code == 0, err - expected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle0000.png" + expected_png_path = ( + tmp_path / "images" / "simple_scenes" / "SquareToCircle" / "0000.png" + ) assert expected_png_path.exists(), "png file not found at " + str(expected_png_path) @pytest.mark.slow -def test_images_are_created_when_png_format_set_for_opengl( +def test_images_are_created_when_png_sequence_format_set_for_opengl( tmp_path, manim_cfg_file, simple_scenes_path, ): - """Test images are created in media directory when --format png is set for opengl""" + """Test OpenGL images are created when --format png-sequence is set.""" scene_name = "SquareToCircle" command = [ sys.executable, @@ -534,24 +560,26 @@ def test_images_are_created_when_png_format_set_for_opengl( "--media_dir", str(tmp_path), "--format", - "png", + "png-sequence", str(simple_scenes_path), scene_name, ] out, err, exit_code = capture(command) assert exit_code == 0, err - expected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle0000.png" + expected_png_path = ( + tmp_path / "images" / "simple_scenes" / "SquareToCircle" / "0000.png" + ) assert expected_png_path.exists(), "png file not found at " + str(expected_png_path) @pytest.mark.slow -def test_images_are_zero_padded_when_zero_pad_set( +def test_png_sequence_images_are_zero_padded( tmp_path, manim_cfg_file, simple_scenes_path, ): - """Test images are zero padded when --format png and --zero_pad n are set""" + """Test PNG-sequence images respect --zero_pad.""" scene_name = "SquareToCircle" command = [ sys.executable, @@ -561,7 +589,7 @@ def test_images_are_zero_padded_when_zero_pad_set( "--media_dir", str(tmp_path), "--format", - "png", + "png-sequence", "--zero_pad", "3", str(simple_scenes_path), @@ -570,22 +598,23 @@ def test_images_are_zero_padded_when_zero_pad_set( out, err, exit_code = capture(command) assert exit_code == 0, err - unexpected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle0.png" + sequence_dir = tmp_path / "images" / "simple_scenes" / "SquareToCircle" + unexpected_png_path = sequence_dir / "0.png" assert not unexpected_png_path.exists(), "non zero padded png file found at " + str( unexpected_png_path, ) - expected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle000.png" + expected_png_path = sequence_dir / "000.png" assert expected_png_path.exists(), "png file not found at " + str(expected_png_path) @pytest.mark.slow -def test_images_are_zero_padded_when_zero_pad_set_for_opengl( +def test_opengl_png_sequence_images_are_zero_padded( tmp_path, manim_cfg_file, simple_scenes_path, ): - """Test images are zero padded when --format png and --zero_pad n are set with the opengl renderer""" + """Test OpenGL PNG-sequence images respect --zero_pad.""" scene_name = "SquareToCircle" command = [ sys.executable, @@ -597,7 +626,7 @@ def test_images_are_zero_padded_when_zero_pad_set_for_opengl( "--media_dir", str(tmp_path), "--format", - "png", + "png-sequence", "--zero_pad", "3", str(simple_scenes_path), @@ -606,12 +635,13 @@ def test_images_are_zero_padded_when_zero_pad_set_for_opengl( out, err, exit_code = capture(command) assert exit_code == 0, err - unexpected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle0.png" + sequence_dir = tmp_path / "images" / "simple_scenes" / "SquareToCircle" + unexpected_png_path = sequence_dir / "0.png" assert not unexpected_png_path.exists(), "non zero padded png file found at " + str( unexpected_png_path, ) - expected_png_path = tmp_path / "images" / "simple_scenes" / "SquareToCircle000.png" + expected_png_path = sequence_dir / "000.png" assert expected_png_path.exists(), "png file not found at " + str(expected_png_path) diff --git a/tests/test_scene_rendering/test_file_writer.py b/tests/test_scene_rendering/test_file_writer.py index 17404453f2..413f668a19 100644 --- a/tests/test_scene_rendering/test_file_writer.py +++ b/tests/test_scene_rendering/test_file_writer.py @@ -1,14 +1,11 @@ import sys -from fractions import Fraction from pathlib import Path -from unittest.mock import Mock import av import numpy as np import pytest from manim import DR, Circle, Create, Scene, Star, tempconfig -from manim.scene.scene_file_writer import SceneFileWriter, to_av_frame_rate from manim.utils.commands import capture, get_video_metadata @@ -178,105 +175,3 @@ def test_unicode_partial_movie(config, tmpdir, simple_scenes_path): _, err, exit_code = capture(command) assert exit_code == 0, err - - -def test_frame_rates(): - assert to_av_frame_rate(25) == Fraction(25, 1) - assert to_av_frame_rate(24.0) == Fraction(24, 1) - assert to_av_frame_rate(23.976) == Fraction(24 * 1000, 1001) - assert to_av_frame_rate(23.98) == Fraction(24 * 1000, 1001) - assert to_av_frame_rate(59.94) == Fraction(60 * 1000, 1001) - - -def _new_file_writer(scene_name: str) -> SceneFileWriter: - renderer = Mock() - renderer.num_plays = 0 - return SceneFileWriter(renderer, scene_name) - - -def test_clean_cache_ignores_hidden_files(config, tmp_path): - # macOS leaves resource forks (._*.mp4) and .DS_Store files in the - # partial movie directory; they must not be counted against - # max_files_cached nor be deleted, see issue #3234. - with tempconfig({"media_dir": tmp_path, "write_to_movie": True}): - writer = _new_file_writer("CacheCleaningScene") - cache_dir = writer.partial_movie_directory - - for name in ["00001.mp4", ".DS_Store", "._00001.mp4"]: - (cache_dir / name).touch() - - config.max_files_cached = 1 - # The hidden files must not count towards the limit: with a single - # real partial movie file cached, nothing must be evicted. - writer.clean_cache() - - assert (cache_dir / "00001.mp4").exists() - assert (cache_dir / ".DS_Store").exists() - assert (cache_dir / "._00001.mp4").exists() - - config.max_files_cached = 0 - writer.clean_cache() - - assert not (cache_dir / "00001.mp4").exists() - assert (cache_dir / ".DS_Store").exists() - assert (cache_dir / "._00001.mp4").exists() - - -def test_flush_cache_directory_ignores_hidden_files(config, tmp_path): - with tempconfig({"media_dir": tmp_path, "write_to_movie": True}): - writer = _new_file_writer("CacheFlushingScene") - cache_dir = writer.partial_movie_directory - - for name in ["00001.mp4", "00002.mp4", ".DS_Store", "._00001.mp4"]: - (cache_dir / name).touch() - (cache_dir / "partial_movie_file_list.txt").touch() - - writer.flush_cache_directory() - - assert not (cache_dir / "00001.mp4").exists() - assert not (cache_dir / "00002.mp4").exists() - assert (cache_dir / "partial_movie_file_list.txt").exists() - assert (cache_dir / ".DS_Store").exists() - assert (cache_dir / "._00001.mp4").exists() - - -def test_clean_cache_tolerates_vanishing_files(config, tmp_path, monkeypatch): - # A file can disappear between listing the directory and unlinking it - # (e.g. Finder removing a transient resource fork); clean_cache must - # not raise FileNotFoundError in that case. - with tempconfig({"media_dir": tmp_path, "write_to_movie": True}): - writer = _new_file_writer("VanishingFileScene") - cache_dir = writer.partial_movie_directory - - survivor = cache_dir / "00001.mp4" - survivor.touch() - ghost = cache_dir / "00002.mp4" - monkeypatch.setattr( - writer, "_cached_partial_movie_files", lambda: [survivor, ghost] - ) - - config.max_files_cached = 0 - writer.clean_cache() - - assert not survivor.exists() - - -def test_clean_cache_does_not_evict_for_vanished_file(config, tmp_path, monkeypatch): - with tempconfig({"media_dir": tmp_path, "write_to_movie": True}): - writer = _new_file_writer("VanishedFileEvictionScene") - cache_dir = writer.partial_movie_directory - - survivors = [cache_dir / f"{index:05}.mp4" for index in range(2)] - for survivor in survivors: - survivor.touch() - ghost = cache_dir / "00002.mp4" - monkeypatch.setattr( - writer, - "_cached_partial_movie_files", - lambda: [*survivors, ghost], - ) - - config.max_files_cached = len(survivors) - writer.clean_cache() - - assert all(survivor.exists() for survivor in survivors) diff --git a/tests/test_scene_rendering/test_parallel_encoding.py b/tests/test_scene_rendering/test_parallel_encoding.py index 474a6fb89f..57a91eebfe 100644 --- a/tests/test_scene_rendering/test_parallel_encoding.py +++ b/tests/test_scene_rendering/test_parallel_encoding.py @@ -6,7 +6,7 @@ import threading import time from pathlib import Path -from unittest.mock import ANY, Mock, call +from unittest.mock import ANY, Mock import av import numpy as np @@ -15,13 +15,69 @@ from manim import FadeIn, Scene, Square, capture, tempconfig from manim._config import config +from manim._config.output import OutputFormat, OutputSpec +from manim._config.output_plan import ( + resolve_media_layout, + resolve_module_name, + resolve_output_plan, + resolve_requested_output_name, +) +from manim._config.video_encoder import resolve_video_encoder from manim.cli.render.commands import render +from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings from manim.utils.exceptions import RerunSceneException _ENCODER_THREAD_PREFIX = "partial-movie-encoder-" +_VIDEO_OUTPUT = OutputSpec( + OutputFormat.MP4, + transparent=False, + save_sections=False, + fallback_to_still=False, +) +_NO_OUTPUT = OutputSpec( + OutputFormat.NONE, + transparent=False, + save_sections=False, + fallback_to_still=False, +) _UNIQUE_PLAYS = 6 _TOTAL_PLAYS = _UNIQUE_PLAYS + 2 + +def _make_writer( + scene_name: str, + output: OutputSpec = _VIDEO_OUTPUT, +) -> SceneFileWriter: + module_name = resolve_module_name(config) + layout = resolve_media_layout( + config, + output, + module_name=module_name, + scene_name=scene_name, + working_directory=Path.cwd(), + ) + plan = resolve_output_plan( + layout, + output, + scene_name=scene_name, + requested_output_name=resolve_requested_output_name(config), + ) + settings = _SceneFileWriterSettings( + plan=plan, + video_encoder=resolve_video_encoder( + output, + width=config.pixel_width, + height=config.pixel_height, + frame_rate=config.frame_rate, + ), + max_inflight_encoders=config.max_inflight_encoders, + encoder_queue_size=config.encoder_queue_size, + max_files_cached=config.max_files_cached, + assets_dir=Path.cwd(), + ) + return SceneFileWriter(settings) + + _SCENE_NAME = "ParallelEncodingCacheScene" _SCENE_SOURCE = textwrap.dedent( f"""\ @@ -127,26 +183,23 @@ def _frame(): return np.zeros((4, 4, 4), dtype=np.uint8) -def _new_encode_job( - tmp_path, - monkeypatch, - name, - stream, - container, - frame_queue_size=8, -): +def _fake_segment_encoder(tmp_path, name): + target = tmp_path / f"{name}.mp4" + encoder = Mock(target=target) + encoder.abort.side_effect = lambda: target.unlink(missing_ok=True) + return encoder + + +def _new_encode_job(tmp_path, name, encoder=None, frame_queue_size=8): from manim.scene.scene_file_writer import _PartialMovieEncodeJob - job = _PartialMovieEncodeJob( - path=tmp_path / f"{name}.mp4", + if encoder is None: + encoder = _fake_segment_encoder(tmp_path, name) + return _PartialMovieEncodeJob( animation_index=0, - stream=Mock(), - container=Mock(), + encoder=encoder, frame_queue_size=frame_queue_size, ) - monkeypatch.setattr(job, "stream", stream) - monkeypatch.setattr(job, "container", container) - return job def _assert_failed_join(job, expected_exception): @@ -162,22 +215,18 @@ def _assert_failed_join(job, expected_exception): def test_encode_failure_propagates_and_drains_bounded_queue( tmp_path, - monkeypatch, manim_caplog, ): expected_exception = RuntimeError("encode failed") encode_failed = threading.Event() - stream = Mock() - container = Mock() + encoder = _fake_segment_encoder(tmp_path, "encode_failure") - def encode(*args): - if args: - encode_failed.set() - raise expected_exception - return [] + def fail_write(*args, **kwargs): + encode_failed.set() + raise expected_exception - stream.encode.side_effect = encode - job = _new_encode_job(tmp_path, monkeypatch, "encode_failure", stream, container) + encoder.write_frame.side_effect = fail_write + job = _new_encode_job(tmp_path, "encode_failure", encoder) job.put(1, _frame()) assert encode_failed.wait(timeout=2), "Encode failure was not triggered" @@ -192,159 +241,61 @@ def fill_queue_and_seal(): assert not producer.is_alive(), "Producer deadlocked on the bounded queue" _assert_failed_join(job, expected_exception) - container.close.assert_called_once_with() - assert "Partial movie file written" not in manim_caplog.text - - -def test_flush_failure_propagates_and_closes_container( - tmp_path, - monkeypatch, - manim_caplog, -): - expected_exception = RuntimeError("flush failed") - stream = Mock() - container = Mock() - - def encode(*args): - if args: - return [] - raise expected_exception - - stream.encode.side_effect = encode - job = _new_encode_job(tmp_path, monkeypatch, "flush_failure", stream, container) - job.put(1, _frame()) - job.seal() - - _assert_failed_join(job, expected_exception) - container.close.assert_called_once_with() + encoder.abort.assert_called_once_with() assert "Partial movie file written" not in manim_caplog.text -def test_close_failure_propagates_after_close_attempt( - tmp_path, - monkeypatch, - manim_caplog, -): - expected_exception = RuntimeError("close failed") - stream = Mock() - stream.encode.return_value = [] - container = Mock() - container.close.side_effect = expected_exception - job = _new_encode_job(tmp_path, monkeypatch, "close_failure", stream, container) +def test_finish_failure_aborts_segment(tmp_path, manim_caplog): + expected_exception = RuntimeError("finish failed") + encoder = _fake_segment_encoder(tmp_path, "finish_failure") + encoder.finish.side_effect = expected_exception + job = _new_encode_job(tmp_path, "finish_failure", encoder) job.put(1, _frame()) job.seal() _assert_failed_join(job, expected_exception) - container.close.assert_called_once_with() + encoder.abort.assert_called_once_with() assert "Partial movie file written" not in manim_caplog.text -def test_encode_failure_precedes_close_failure_and_removes_partial( - tmp_path, - monkeypatch, - manim_caplog, -): +def test_encode_failure_precedes_abort_failure(tmp_path, manim_caplog): expected_exception = RuntimeError("encode failed") - close_exception = RuntimeError("close failed") - stream = Mock() - container = Mock() - - def encode(*args): - if args: - raise expected_exception - return [] - - stream.encode.side_effect = encode - container.close.side_effect = close_exception - job = _new_encode_job( - tmp_path, - monkeypatch, - "encode_and_close_failure", - stream, - container, - ) - job.path.write_bytes(b"stale") + abort_exception = RuntimeError("abort failed") + encoder = _fake_segment_encoder(tmp_path, "encode_and_abort_failure") + encoder.write_frame.side_effect = expected_exception + encoder.abort.side_effect = abort_exception + job = _new_encode_job(tmp_path, "encode_and_abort_failure", encoder) job.put(1, _frame()) job.seal() _assert_failed_join(job, expected_exception) - container.close.assert_called_once_with() - assert not job.path.exists() + assert "Failed to clean up incomplete segment" in manim_caplog.text + assert "abort failed" in manim_caplog.text assert "Partial movie file written" not in manim_caplog.text -def test_partial_cleanup_failure_does_not_mask_encode_failure( - tmp_path, - monkeypatch, - manim_caplog, -): - expected_exception = RuntimeError("encode failed") - cleanup_exception = PermissionError("cannot remove partial") - stream = Mock() - container = Mock() - - def encode(*args): - if args: - raise expected_exception - return [] - - stream.encode.side_effect = encode - job = _new_encode_job(tmp_path, monkeypatch, "cleanup_failure", stream, container) - job.path.write_bytes(b"stale") - job.put(1, _frame()) - job.seal() - unlink = Mock(side_effect=cleanup_exception) - monkeypatch.setattr(Path, "unlink", unlink) - - _assert_failed_join(job, expected_exception) - - unlink.assert_called_once_with(missing_ok=True) - assert "Failed to remove incomplete partial movie file" in manim_caplog.text - assert "cannot remove partial" in manim_caplog.text - assert "Partial movie file written" not in manim_caplog.text - - -def test_successful_encode_job_logs_partial_movie_written( - tmp_path, - monkeypatch, - manim_caplog, -): - stream = Mock() - stream.encode.return_value = [] - container = Mock() - job = _new_encode_job(tmp_path, monkeypatch, "encode_success", stream, container) +def test_successful_encode_job_logs_partial_movie_written(tmp_path, manim_caplog): + encoder = _fake_segment_encoder(tmp_path, "encode_success") + job = _new_encode_job(tmp_path, "encode_success", encoder) job.put(1, _frame()) job.seal() job.join() - container.close.assert_called_once_with() + encoder.write_frame.assert_called_once_with(ANY, repeat=1) + encoder.finish.assert_called_once_with() + encoder.abort.assert_not_called() assert "Partial movie file written" in manim_caplog.text assert not _alive_encoder_threads() -def test_write_frame_fails_fast_after_encoder_failure( - config, - tmp_path, - monkeypatch, -): - from manim.scene.scene_file_writer import SceneFileWriter - +def test_write_frame_fails_fast_after_encoder_failure(config, tmp_path): expected_exception = RuntimeError("encode failed") - stream = Mock() - container = Mock() - - def encode(*args): - if args: - raise expected_exception - return [] - - stream.encode.side_effect = encode + encoder = _fake_segment_encoder(tmp_path, "fail_fast") + encoder.write_frame.side_effect = expected_exception config.media_dir = str(tmp_path) - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "FailFastScene") - job = _new_encode_job(tmp_path, monkeypatch, "fail_fast", stream, container) + writer = _make_writer("FailFastScene") + job = _new_encode_job(tmp_path, "fail_fast", encoder) job.path.write_bytes(b"stale") writer._current_encode_job = job @@ -364,10 +315,8 @@ def encode(*args): assert not job.path.exists() assert not _alive_encoder_threads() finally: - # An assertion failure above must not leave an unsealed non-daemon - # worker behind: it would hang pytest at exit. if writer._current_encode_job is not None: - job.seal() + job.abort() writer._current_encode_job = None job.thread.join(timeout=5) @@ -383,15 +332,14 @@ def test_frame_queue_configuration( encoder_queue_size, expected_queue_size, ): - from manim.scene.scene_file_writer import SceneFileWriter - config.max_inflight_encoders = max_inflight_encoders config.encoder_queue_size = encoder_queue_size - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "FrameQueueSizeScene") + writer = _make_writer("FrameQueueSizeScene") - writer.open_partial_movie_stream(tmp_path / "partial.mp4") + writer.open_partial_movie_stream( + animation_index=0, + file_path=tmp_path / "partial.mp4", + ) job = writer._current_encode_job assert job is not None assert job.queue.maxsize == expected_queue_size @@ -401,18 +349,51 @@ def test_frame_queue_configuration( assert not _alive_encoder_threads() +def test_pool_settings_are_not_read_from_mutated_config(config, tmp_path): + config.max_inflight_encoders = 2 + config.encoder_queue_size = 3 + writer = _make_writer("CapturedPoolSettingsScene") + config.max_inflight_encoders = 1 + config.encoder_queue_size = 9 + + writer.open_partial_movie_stream( + animation_index=0, + file_path=tmp_path / "partial.mp4", + ) + job = writer._current_encode_job + assert job is not None + assert job.queue.maxsize == 3 + + writer.close_partial_movie_stream() + assert writer._inflight_encode_jobs == [job] + writer.join_all_encode_jobs() + assert not _alive_encoder_threads() + + +def test_cache_limit_is_not_read_from_mutated_config(config, monkeypatch): + config.max_files_cached = -1 + writer = _make_writer("CapturedCacheLimitScene") + config.max_files_cached = 0 + writer.combine_to_movie = Mock() + prune = Mock() + monkeypatch.setattr( + "manim.scene.scene_file_writer.prune_segment_cache", + prune, + ) + + writer.finish() + + prune.assert_called_once_with(writer.partial_movie_directory, -1) + + @pytest.mark.parametrize("max_inflight_encoders", [1, 2, 3]) def test_close_partial_movie_stream_respects_cap_and_joins_fifo( config, tmp_path, max_inflight_encoders, ): - from manim.scene.scene_file_writer import SceneFileWriter - config.max_inflight_encoders = max_inflight_encoders - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "EncoderCapScene") + writer = _make_writer("EncoderCapScene") jobs = [Mock(path=tmp_path / f"partial_{index}.mp4") for index in range(3)] for index, job in enumerate(jobs): @@ -443,14 +424,10 @@ def test_close_partial_movie_stream_respects_cap_and_joins_fifo( def test_cap_join_failure_drains_all_inflight_jobs(config, tmp_path): - from manim.scene.scene_file_writer import SceneFileWriter - primary_exception = RuntimeError("first join failed") secondary_exception = RuntimeError("second join failed") config.max_inflight_encoders = 3 - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "EncoderCapFailureScene") + writer = _make_writer("EncoderCapFailureScene") jobs = [Mock(path=tmp_path / f"partial_{index}.mp4") for index in range(3)] jobs[0].join.side_effect = primary_exception jobs[1].join.side_effect = secondary_exception @@ -473,15 +450,11 @@ def test_cap_join_failure_drains_all_inflight_jobs(config, tmp_path): def test_is_already_cached_joins_same_path_inflight_job(config, tmp_path): - from manim.scene.scene_file_writer import SceneFileWriter - - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "CachedInflightScene") + writer = _make_writer("CachedInflightScene") hash_invocation = "same_path_hash" path = ( writer.partial_movie_directory - / f"{hash_invocation}{config['movie_file_extension']}" + / f"{hash_invocation}{writer.output_spec.segment_extension}" ) job = Mock(path=path) writer._inflight_encode_jobs.append(job) @@ -495,16 +468,12 @@ def test_is_already_cached_joins_same_path_inflight_job(config, tmp_path): def test_same_path_join_failure_drains_unrelated_jobs(config, tmp_path): - from manim.scene.scene_file_writer import SceneFileWriter - expected_exception = RuntimeError("same-path join failed") - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "CachedInflightFailureScene") + writer = _make_writer("CachedInflightFailureScene") hash_invocation = "failing_same_path_hash" path = ( writer.partial_movie_directory - / f"{hash_invocation}{config['movie_file_extension']}" + / f"{hash_invocation}{writer.output_spec.segment_extension}" ) unrelated_job = Mock(path=tmp_path / "unrelated.mp4") same_path_job = Mock(path=path) @@ -524,17 +493,13 @@ def test_same_path_join_failure_drains_unrelated_jobs(config, tmp_path): def test_open_partial_movie_stream_joins_same_path_inflight_job(config, tmp_path): - from manim.scene.scene_file_writer import SceneFileWriter - - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "OpenInflightScene") + writer = _make_writer("OpenInflightScene") path = tmp_path / "same_path.mp4" inflight_job = Mock(path=path) writer._inflight_encode_jobs.append(inflight_job) writer._inflight_by_path[str(path)] = inflight_job - writer.open_partial_movie_stream(file_path=path) + writer.open_partial_movie_stream(animation_index=0, file_path=path) current_job = writer._current_encode_job assert current_job is not None try: @@ -554,12 +519,8 @@ def test_finish_propagates_join_failure_and_clears_inflight_state( tmp_path, monkeypatch, ): - from manim.scene.scene_file_writer import SceneFileWriter - expected_exception = RuntimeError("join failed") - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "JoinFailureScene") + writer = _make_writer("JoinFailureScene") failing_job = Mock(path=tmp_path / "failing.mp4") failing_job.join.side_effect = expected_exception succeeding_job = Mock(path=tmp_path / "succeeding.mp4") @@ -581,18 +542,12 @@ def test_finish_propagates_join_failure_and_clears_inflight_state( def _new_writer(config, tmp_path, scene_name): - from manim.scene.scene_file_writer import SceneFileWriter - config.media_dir = str(tmp_path) - renderer = Mock() - renderer.num_plays = 0 - return SceneFileWriter(renderer, scene_name) + return _make_writer(scene_name) -def _healthy_current_job(tmp_path, monkeypatch, name): - stream = Mock() - stream.encode.return_value = [] - job = _new_encode_job(tmp_path, monkeypatch, name, stream, Mock()) +def _healthy_current_job(tmp_path, name): + job = _new_encode_job(tmp_path, name) job.path.write_bytes(b"stale") return job @@ -605,11 +560,10 @@ def _add_inflight_job(writer, job): def test_abort_encode_jobs_unlinks_current_and_drains_inflight( config, tmp_path, - monkeypatch, manim_caplog, ): writer = _new_writer(config, tmp_path, "AbortScene") - job = _healthy_current_job(tmp_path, monkeypatch, "abort_current") + job = _healthy_current_job(tmp_path, "abort_current") writer._current_encode_job = job failing_inflight = Mock(path=tmp_path / "inflight.mp4") failing_inflight.join.side_effect = RuntimeError("in-flight join failed") @@ -655,7 +609,7 @@ def test_abort_encode_jobs_cleanup_failure_logs_warning( manim_caplog, ): writer = _new_writer(config, tmp_path, "AbortCleanupFailureScene") - job = _healthy_current_job(tmp_path, monkeypatch, "abort_cleanup_failure") + job = _healthy_current_job(tmp_path, "abort_cleanup_failure") writer._current_encode_job = job unlink = Mock(side_effect=PermissionError("cannot remove partial")) monkeypatch.setattr(Path, "unlink", unlink) @@ -664,19 +618,15 @@ def test_abort_encode_jobs_cleanup_failure_logs_warning( assert writer._current_encode_job is None unlink.assert_called_once_with(missing_ok=True) - assert "Failed to remove incomplete partial movie file" in manim_caplog.text + assert "Failed to clean up incomplete segment" in manim_caplog.text assert "cannot remove partial" in manim_caplog.text assert "Discarded partial movie file" not in manim_caplog.text assert not _alive_encoder_threads() def test_abort_encode_jobs_noop_on_dry_run_writer(config): - from manim.scene.scene_file_writer import SceneFileWriter - with tempconfig({"dry_run": True}): - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "DryRunAbortScene") + writer = _make_writer("DryRunAbortScene", _NO_OUTPUT) writer.abort_encode_jobs() writer.abort_encode_jobs(reraise_encoder_failures=True) @@ -720,18 +670,12 @@ def construct(): assert writer._inflight_by_path == {} -def test_rerun_propagates_failed_current_job(config, tmp_path, monkeypatch): +def test_rerun_propagates_failed_current_job(config, tmp_path): expected_exception = RuntimeError("encode failed") - stream = Mock() - - def encode(*args): - if args: - raise expected_exception - return [] - - stream.encode.side_effect = encode + encoder = _fake_segment_encoder(tmp_path, "rerun_current") + encoder.write_frame.side_effect = expected_exception writer = _new_writer(config, tmp_path, "RerunCurrentFailureScene") - job = _new_encode_job(tmp_path, monkeypatch, "rerun_current", stream, Mock()) + job = _new_encode_job(tmp_path, "rerun_current", encoder) job.path.write_bytes(b"stale") writer._current_encode_job = job job.put(1, _frame()) @@ -762,7 +706,7 @@ def construct(): # An assertion failure above must not leave an unsealed non-daemon # worker behind: it would hang pytest at exit. if writer._current_encode_job is not None: - job.seal() + job.abort() writer._current_encode_job = None job.thread.join(timeout=5) @@ -977,33 +921,17 @@ def test_parallel_encoding_output_matches_serial(tmp_path): ) -def test_encode_job_repeats_frame_num_frames_times(tmp_path): - """``put(n, frame)`` must encode the frame n times and mux every packet. +def test_encode_job_forwards_frame_repeat(tmp_path): + encoder = _fake_segment_encoder(tmp_path, "freeze_frame") + job = _new_encode_job(tmp_path, "freeze_frame", encoder) - Every other test in this module uses ``num_frames=1``, leaving the - repetition loop in ``_encode_and_write_frame`` uncovered. - """ - from manim.scene.scene_file_writer import _PartialMovieEncodeJob - - packet = object() - stream = Mock() - stream.encode.side_effect = lambda *args: [packet] if args else [] - container = Mock() - - job = _PartialMovieEncodeJob( - path=tmp_path / "freeze_frame.mp4", - animation_index=0, - container=container, - stream=stream, - frame_queue_size=8, - ) - job.put(3, _frame()) + frame = _frame() + job.put(3, frame) job.seal() job.join() - # Three encode calls with a frame, then the argless flush. - assert stream.encode.call_args_list == [call(ANY)] * 3 + [call()] - assert container.mux.call_args_list == [call(packet)] * 3 + encoder.write_frame.assert_called_once_with(frame, repeat=3) + encoder.finish.assert_called_once_with() def test_is_already_cached_false_after_joining_failed_path(config, tmp_path): @@ -1012,16 +940,12 @@ def test_is_already_cached_false_after_joining_failed_path(config, tmp_path): The existing same-path test asserts the join happens but never checks the return value. """ - from manim.scene.scene_file_writer import SceneFileWriter - with tempconfig({"media_dir": str(tmp_path)}): - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "CachedReturnScene") + writer = _make_writer("CachedReturnScene") hash_invocation = "missing_partial_hash" path = ( writer.partial_movie_directory - / f"{hash_invocation}{config['movie_file_extension']}" + / f"{hash_invocation}{writer.output_spec.segment_extension}" ) job = Mock(path=path) writer._inflight_encode_jobs.append(job) @@ -1032,59 +956,52 @@ def test_is_already_cached_false_after_joining_failed_path(config, tmp_path): def test_is_already_cached_true_when_partial_exists(config, tmp_path): - from manim.scene.scene_file_writer import SceneFileWriter - with tempconfig({"media_dir": str(tmp_path)}): - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "CachedReturnScene") + writer = _make_writer("CachedReturnScene") hash_invocation = "present_partial_hash" path = ( writer.partial_movie_directory - / f"{hash_invocation}{config['movie_file_extension']}" + / f"{hash_invocation}{writer.output_spec.segment_extension}" ) + path.parent.mkdir(parents=True) path.write_bytes(b"cached partial") assert writer.is_already_cached(hash_invocation) is True def test_close_partial_movie_stream_without_open_stream_raises(config, tmp_path): - from manim.scene.scene_file_writer import SceneFileWriter - with tempconfig({"media_dir": str(tmp_path)}): - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "GuardScene") + writer = _make_writer("GuardScene") with pytest.raises(RuntimeError, match="without an open partial"): writer.close_partial_movie_stream() def test_open_partial_movie_stream_without_path_raises(config, tmp_path): - from manim.scene.scene_file_writer import SceneFileWriter - with tempconfig({"media_dir": str(tmp_path)}): - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "GuardScene") + writer = _make_writer("GuardScene") writer.partial_movie_files = [None] with pytest.raises(RuntimeError, match="partial movie file path"): - writer.open_partial_movie_stream() + writer.open_partial_movie_stream(animation_index=0) -def test_write_frame_without_open_stream_drops_frame(config, tmp_path): - """Interactive OpenGL emits frames with no open stream; they are dropped. +def test_open_partial_movie_stream_rejects_nested_current_segment(config, tmp_path): + with tempconfig({"media_dir": str(tmp_path)}): + writer = _make_writer("GuardScene") + writer._current_encode_job = Mock() + + with pytest.raises(RuntimeError, match="another segment is still open"): + writer.open_partial_movie_stream( + animation_index=1, + file_path=tmp_path / "nested.mp4", + ) - ``write_to_movie()`` is true under the default test config, so the call - reaches the drop branch in ``write_frame``. - """ - from manim.scene.scene_file_writer import SceneFileWriter +def test_write_frame_without_open_stream_drops_frame(config, tmp_path): + """Frames outside an open segment are ignored by the transitional scheduler.""" with tempconfig({"media_dir": str(tmp_path)}): - renderer = Mock() - renderer.num_plays = 0 - writer = SceneFileWriter(renderer, "DropFrameScene") + writer = _make_writer("DropFrameScene") assert writer._current_encode_job is None # Must not raise and must not create a job. diff --git a/tests/test_scene_rendering/test_play_logic.py b/tests/test_scene_rendering/test_play_logic.py index 4aaae3c8ad..cb5fdd1b46 100644 --- a/tests/test_scene_rendering/test_play_logic.py +++ b/tests/test_scene_rendering/test_play_logic.py @@ -112,15 +112,32 @@ def test_t_values_with_cached_data(using_temp_config): assert scene.update_to_time.call_count == 10 -def test_t_values_save_last_frame(config, using_temp_config): - """Test that there is only one t value handled when only saving the last frame""" - config.save_last_frame = True +@pytest.mark.parametrize( + "still_config", + [{"save_last_frame": True}, {"format": "png"}], +) +def test_t_values_save_last_frame(config, using_temp_config, still_config): + """Still output fast-forwards each play and only evaluates its final state.""" + config.update(still_config) scene = SquareToCircle() scene.update_to_time = Mock() scene.render() scene.update_to_time.assert_called_once_with(1) +def test_png_sequence_evaluates_every_frame(config, using_temp_config): + config.format = "png-sequence" + scene = SquareToCircle() + scene.update_to_time = Mock() + + scene.render() + + assert scene.update_to_time.call_count == config.frame_rate + assert scene.renderer.file_writer.frame_count == config.frame_rate + assert scene.renderer.file_writer.final_file_path.is_dir() + assert (scene.renderer.file_writer.final_file_path / "0000.png").is_file() + + def test_animate_with_changed_custom_attribute(using_temp_config): """Test that animating the change of a custom attribute using the animate syntax works correctly. diff --git a/tests/test_scene_rendering/test_video_segment_profiles.py b/tests/test_scene_rendering/test_video_segment_profiles.py new file mode 100644 index 0000000000..9ff96a15c3 --- /dev/null +++ b/tests/test_scene_rendering/test_video_segment_profiles.py @@ -0,0 +1,204 @@ +from __future__ import annotations + +from fractions import Fraction +from pathlib import Path + +import av +import numpy as np +import pytest +from av.codec.context import CodecContext + +from manim import tempconfig +from manim._config import config +from manim._config.output import OutputFormat, OutputSpec +from manim._config.output_plan import resolve_media_layout, resolve_output_plan +from manim._config.video_encoder import resolve_video_encoder +from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings + +_WIDTH = 16 +_HEIGHT = 12 +_FRAME_RATE = 5 +_FRAME_COUNT = 3 + + +def _output(format: str, transparent: bool) -> OutputSpec: + return OutputSpec( + format=OutputFormat(format), + transparent=transparent, + save_sections=False, + fallback_to_still=False, + ) + + +def _writer(output: OutputSpec) -> SceneFileWriter: + layout = resolve_media_layout( + config, + output, + module_name="segment_profiles", + scene_name="SegmentProfileScene", + working_directory=Path.cwd(), + ) + plan = resolve_output_plan( + layout, + output, + scene_name="SegmentProfileScene", + requested_output_name=None, + ) + video_encoder = resolve_video_encoder( + output, + width=config.pixel_width, + height=config.pixel_height, + frame_rate=config.frame_rate, + codec=config.video_codec, + pixel_format=config.pixel_format, + options=config.video_encoder_options, + ) + settings = _SceneFileWriterSettings( + plan=plan, + video_encoder=video_encoder, + max_inflight_encoders=config.max_inflight_encoders, + encoder_queue_size=config.encoder_queue_size, + max_files_cached=config.max_files_cached, + assets_dir=Path.cwd(), + ) + return SceneFileWriter(settings) + + +def _asymmetric_rgba_frame() -> np.ndarray: + frame = np.empty((_HEIGHT, _WIDTH, 4), dtype=np.uint8) + frame[: _HEIGHT // 2, : _WIDTH // 2] = [255, 0, 0, 32] + frame[: _HEIGHT // 2, _WIDTH // 2 :] = [0, 255, 0, 96] + frame[_HEIGHT // 2 :, : _WIDTH // 2] = [0, 0, 255, 160] + frame[_HEIGHT // 2 :, _WIDTH // 2 :] = [255, 255, 255, 224] + return frame + + +def _decode_frames( + path: Path, + *, + transparent_vp9: bool, +) -> tuple[str, str, Fraction | None, list[np.ndarray], int]: + with av.open(path) as container: + stream = container.streams.video[0] + codec = stream.codec_context.name + pixel_format = stream.codec_context.format.name + rate = stream.average_rate + audio_streams = len(container.streams.audio) + + if transparent_vp9: + decoder = CodecContext.create("libvpx-vp9", "r") + decoded = [] + for packet in container.demux(video=0): + decoded.extend(decoder.decode(packet)) + pixel_format = decoded[0].format.name + else: + decoded = list(container.decode(video=0)) + + frames = [frame.to_ndarray(format="rgba") for frame in decoded] + return codec, pixel_format, rate, frames, audio_streams + + +@pytest.mark.parametrize( + ( + "format", + "transparent", + "segment_extension", + "codec", + "pixel_format", + ), + [ + ("mp4", False, ".mp4", "h264", "yuv420p"), + ("mov", False, ".mov", "h264", "yuv420p"), + ("mov", True, ".mov", "qtrle", "argb"), + ("webm", False, ".webm", "vp9", "yuv420p"), + ("webm", True, ".webm", "vp9", "yuva420p"), + ], +) +def test_cached_segment_profile_and_pixel_orientation( + tmp_path, + format, + transparent, + segment_extension, + codec, + pixel_format, +): + output = _output(format, transparent) + target = tmp_path / f"segment{segment_extension}" + source = _asymmetric_rgba_frame() + + with tempconfig( + { + "media_dir": tmp_path, + "pixel_width": _WIDTH, + "pixel_height": _HEIGHT, + "frame_rate": _FRAME_RATE, + }, + ): + writer = _writer(output) + writer.open_partial_movie_stream(animation_index=0, file_path=target) + writer.write_frame(source, repeat=_FRAME_COUNT) + writer.close_partial_movie_stream() + writer.join_all_encode_jobs() + + ( + actual_codec, + actual_pixel_format, + actual_rate, + decoded_frames, + audio_streams, + ) = _decode_frames( + target, + transparent_vp9=format == "webm" and transparent, + ) + + assert output.segment_extension == segment_extension + assert actual_codec == codec + assert actual_pixel_format == pixel_format + assert actual_rate == Fraction(_FRAME_RATE, 1) + assert len(decoded_frames) == _FRAME_COUNT + assert audio_streams == 0 + + sample_points = ( + ((_HEIGHT // 4, _WIDTH // 4), np.array([255, 0, 0, 32])), + ((_HEIGHT // 4, 3 * _WIDTH // 4), np.array([0, 255, 0, 96])), + ((3 * _HEIGHT // 4, _WIDTH // 4), np.array([0, 0, 255, 160])), + ((3 * _HEIGHT // 4, 3 * _WIDTH // 4), np.array([255, 255, 255, 224])), + ) + decoded = decoded_frames[0] + for (row, column), expected in sample_points: + expected = expected.copy() + if not transparent: + expected[3] = 255 + np.testing.assert_allclose(decoded[row, column], expected, atol=15) + + +def test_configured_encoder_profile_is_used_for_cached_segment(tmp_path): + output = _output("mov", transparent=False) + target = tmp_path / "custom.mov" + + with tempconfig( + { + "media_dir": tmp_path, + "pixel_width": _WIDTH, + "pixel_height": _HEIGHT, + "frame_rate": _FRAME_RATE, + "video_codec": "qtrle", + "pixel_format": "argb", + "video_encoder_options": {}, + }, + ): + writer = _writer(output) + writer.open_partial_movie_stream(animation_index=0, file_path=target) + writer.write_frame(_asymmetric_rgba_frame()) + writer.close_partial_movie_stream() + writer.join_all_encode_jobs() + + codec, pixel_format, rate, frames, audio_streams = _decode_frames( + target, + transparent_vp9=False, + ) + assert codec == "qtrle" + assert pixel_format == "argb" + assert rate == Fraction(_FRAME_RATE, 1) + assert len(frames) == 1 + assert audio_streams == 0