Skip to main content
  • Home
  • Development
  • Documentation
  • Donate
  • Operational login
  • Browse the archive

swh logo
SoftwareHeritage
Software
Heritage
Archive
Features
  • Search

  • Downloads

  • Save code now

  • Add forge now

  • Help

https://github.com/brownvc/deep-synth
31 March 2020, 06:46:23 UTC
  • Code
  • Branches (1)
  • Releases (0)
  • Visits
Revision b800e11290b763b58e7d3b30329769a7b77cd12a authored by kwang-ether on 14 June 2019, 23:53:57 UTC, committed by kwang-ether on 14 June 2019, 23:53:57 UTC
remove csv
1 parent 79eaa7f
  • Files
  • Changes
    • Branches
    • Releases
    • HEAD
    • refs/heads/master
    • b800e11290b763b58e7d3b30329769a7b77cd12a
    No releases to show
  • 291f7df
  • /
  • deep-synth
  • /
  • location_dataset.py
Raw File Download
Take a new snapshot of a software origin

If the archived software origin currently browsed is not synchronized with its upstream version (for instance when new commits have been issued), you can explicitly request Software Heritage to take a new snapshot of it.

Use the form below to proceed. Once a request has been submitted and accepted, it will be processed as soon as possible. You can then check its processing state by visiting this dedicated page.
swh spinner

Processing "take a new snapshot" request ...

To reference or cite the objects present in the Software Heritage archive, permalinks based on SoftWare Hash IDentifiers (SWHIDs) must be used.
Select below a type of object currently browsed in order to display its associated SWHID and permalink.

  • revision
  • directory
  • content
  • snapshot
origin badgerevision badge
swh:1:rev:b800e11290b763b58e7d3b30329769a7b77cd12a
origin badgedirectory badge
swh:1:dir:87b9fce6556ee1dd9155a27d000889cbaac88195
origin badgecontent badge
swh:1:cnt:9c71cf02435b909fca5cbde26039200ebc3dfca8
origin badgesnapshot badge
swh:1:snp:0f10b5007a9962ed82323ed2242cf08ba5544645

This interface enables to generate software citations, provided that the root directory of browsed objects contains a citation.cff or codemeta.json file.
Select below a type of object currently browsed in order to generate citations for them.

  • revision
  • directory
  • content
  • snapshot
Generate software citation in BibTex format (requires biblatex-software package)
Generating citation ...
Generate software citation in BibTex format (requires biblatex-software package)
Generating citation ...
Generate software citation in BibTex format (requires biblatex-software package)
Generating citation ...
Generate software citation in BibTex format (requires biblatex-software package)
Generating citation ...
Tip revision: b800e11290b763b58e7d3b30329769a7b77cd12a authored by kwang-ether on 14 June 2019, 23:53:57 UTC
remove csv
Tip revision: b800e11
location_dataset.py
from torch.utils import data
import torch
import random
import math
from data import ObjectCategories, RenderedScene, RenderedComposite

class LocationDataset():
    """
    Dataset for training/testing the location/category network
    """
    def __init__(self, data_dir, data_root_dir, scene_indices=(0,4000), p_auxiliary=0.7, seed=None, ablation=None):
        """
        Parameters
        ----------
        data_root_dir (String): root dir where all data lives
        data_dir (String): directory where this dataset lives (relative to data_root_dir)
        scene_indices (tuple[int, int]): list of indices of scenes (in data_dir) that are considered part of this set
        p_auxiliary (int): probability that a auxiliary category is chosen
            Note that since (existing centroid) is sparse, it is actually treated as non-auxiliary when sampling
        seed (int or None, optional): random seed, to replicate stuff, if set
        ablation (string or None, optional): see data.RenderedComposite.get_composite, and paper
        """
        self.category_map = ObjectCategories()
        self.seed = seed
        self.data_dir = data_dir
        self.data_root_dir = data_root_dir
        self.scene_indices = scene_indices
        self.p_auxiliary = p_auxiliary
        self.ablation = ablation

    def __len__(self):
        return self.scene_indices[1]-self.scene_indices[0]

    def __getitem__(self,index):
        if self.seed:
            random.seed(self.seed)

        i = index+self.scene_indices[0]
        scene = RenderedScene(i, self.data_dir, self.data_root_dir)
        composite = scene.create_composite() #get empty composite
        
        #Select a subset of objects randomly. Number of objects is uniformly
        #distributed between [0, total_number_of_objects]
        #Doors and windows do not count here
        object_nodes = scene.object_nodes
        num_objects = random.randint(0, len(object_nodes))
        
        num_categories = len(scene.categories)
        OUTSIDE = num_categories + 2
        EXISTING = num_categories + 1
        NOTHING = num_categories

        centroids = []
        existing_categories = torch.zeros(num_categories)
        future_categories = torch.zeros(num_categories)
        #Process existing objects
        for i in range(num_objects):
            #Add object to composite
            node = object_nodes[i]
            composite.add_node(node)
            #Add existing centroids
            xmin, _, ymin, _ = node["bbox_min"]
            xmax, _, ymax, _ = node["bbox_max"]
            centroids.append(((xmin+xmax)/2, (ymin+ymax)/2, EXISTING))
            existing_categories[node["category"]] += 1
        
        inputs = composite.get_composite(ablation=self.ablation)
        size = inputs.shape[1]
        #Process removed objects
        for i in range(num_objects, len(object_nodes)):
            node = object_nodes[i]
            xmin, _, ymin, _ = node["bbox_min"]
            xmax, _, ymax, _ = node["bbox_max"]
            centroids.append(((xmin+xmax)/2, (ymin+ymax)/2, node["category"]))
            future_categories[node["category"]] += 1
        
        resample = True
        if random.uniform(0,1) > self.p_auxiliary: #Sample an object at random
            x,y,output_centroid = random.choice(centroids)
            x = int(x)
            y = int(y)
        else: #Or sample an auxiliary category
            while resample:
                x, y = random.randint(0,511), random.randint(0,511) #Should probably remove this hardcoded size at somepoint
                good = True
                for (xc, yc, _) in centroids: 
                    #We don't want to sample an empty space that's too close to a centroid
                    #That is, if it can fall within the ground truth attention mask of an object
                    if x-4 < xc < x+5 and y-4 < yc < y+5:
                        good = False 

                if good:
                    if not inputs[0][x][y]:
                        output_centroid = OUTSIDE #Outside of room
                        if random.uniform(0,1) > 0.8: #Just some hardcoded stuff simple outside room is learned very easily
                            resample = False
                    else:
                        output_centroid = NOTHING #Inside of room
                        resample = False

        #Attention mask
        xmin = max(x - 4, 0)
        xmax = min(x + 5, size)
        ymin = max(y - 4, 0)
        ymax = min(y + 5, size)
        inputs[-1, xmin:xmax, ymin:ymax] = 1 #Create attention mask
        
        #Compute weight for L_Global
        #If some objects in this cateogory are removed, weight is zero
        #Otherwise linearly scaled based on completeness of the room
        #See paper for details
        penalty = torch.zeros(num_categories)
        penalty[future_categories==0] = num_objects/len(object_nodes)

        return inputs, output_centroid, existing_categories, penalty
The diff you're trying to view is too large. Only the first 1000 changed files have been loaded.
Showing with 0 additions and 0 deletions (0 / 0 diffs computed)
swh spinner

Computing file changes ...

back to top

Software Heritage — Copyright (C) 2015–2025, The Software Heritage developers. License: GNU AGPLv3+.
The source code of Software Heritage itself is available on our development forge.
The source code files archived by Software Heritage are available under their own copyright and licenses.
Terms of use: Archive access, API— Content policy— Contact— JavaScript license information— Web API