R - Copy out selected LiDAR files into a different folder

Before this, Please refer to the post "LiDAR file Processing in R, C#, MS-DOS, ArcGIS & LAStools"

Once you export the selected attributes into a text file. Then open a text file in a text pad and save that particular column in different text file.

Manually, it will be very difficult to copy out selected files of 2548 files. Here is the R program which helps to copy out all these selected files into a separate folder automatically.

work_dir <- "K:/Resource Info/20140312_CHSTP-Backersfield to Palmdale/Rasters/LIDAR/From_GeoDigital_July_2012/"

setwd(work_dir)

las_file_nos <- readLines("Selected_LAS.csv")

las_file_nos <- paste0(las_file_nos)

las_file_names <- list.files("LAS/", pattern = "BP_")

las_present <- las_file_nos %in% las_file_names

for (eachFile in 1:length(las_present)) {

  if (las_present[eachFile]) {

    file.copy(paste0("LAS/", las_file_nos[eachFile]), 

              "LAS/Output folder/")

  }

}

Web Services (OGC, WMS, WMTS, WCS, WFS, CSW)

WFS = Web Feature Services
WMS = Web Map Services

R - Extract several zipped files into a single folder

# read zipped files and extract tif or jpeg files from the bottom most folder

I have downloaded 4200 1' Orthoimagery raster files from USGS. Each zipped folder has several subfolder and then you will find the raster files.

The goal is to get raster files from all the zipped folder into a single folder.

setwd("I:/J_Prashant's External HD/High Speed Rail/GIS/Bulk Data/Bulk data/Firm Panels/")

output_location <- "output_folder"
dir.create(output_location)

zipped_files <- list.files()
zipped_files <- zipped_files[grepl(".zip", zipped_files)]

for (eachFile in zipped_files) {
  unzip(zipfile = eachFile, exdir = output_location, junkpaths = TRUE)
}

LiDAR file Processing in R, C#, MS-DOS, ArcGIS & LAStools

LiDAR files always comes in .LAS or .LAZ format. These are some of LiDAR files out of 2548 LiDAR files. LiDAR files are the point files. The first question comes to an user's mind How to see the coverage of each of the LiDAR files?

Now open an ArcMAP Document-- Open ArctoolBox --Click on 3D Analyst tools --Click on Conversion -- Click on From File -- Double click on Point File Information

Once the geoprocessing completes then you will get a polygon shapefile in a Table of content of your ArcMAP document.

More Closer look of these LiDAR tiles

If you open an attribute table of this polygon shapefile then you will find the LAS file names are associated with each of these LiDAR tiles.  The attribute table will going to show LiDAR file Names, Number of points in each LiDAR files, Average Point spacing, Lowest and Highest Elevation.

Let's say if i want to get all the LiDAR files with in the red boundary of a project area in a separate folder. Then, First we need to select the attributes with in the red boundary and export the selected rows in a separate shapefile.

Export these selected attributes into a text file and open in a textpad. Now Please refer to "Copy out selected LiDAR files into a different folder in R"

R - Combine text files vertically

Lets say if you want to combine some selected xyz files vertically from a single folder. Then this task can be done in R. Make sure to check whether that particular file is in comma formatted or Space  formatted and you need to make those changes accordingly.

workDir <- "Q:/Jellys Ferry Drainage/LiDAR_Before_Projection/"
setwd(workDir)
inData1 <- read.csv("1132n0266e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData2 <- read.csv("1134n0264e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData3 <- read.csv("1134n0266e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData4 <- read.csv("1134n0268e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData5 <- read.csv("1136n0264e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData6 <- read.csv("1136n0266e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData7 <- read.csv("1136n0268e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData8 <- read.csv("1138n0264e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData9 <- read.csv("1138n0266e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData10 <- read.csv("1138n0268e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData11 <- read.csv("1140n0264e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
inData12 <- read.csv("1140n0266e5k_XYZ.txt", header=FALSE, sep=",", as.is=TRUE)
outData <- rbind(inData1, inData2, inData3, inData4, inData5, inData6, inData7, inData8, inData9, inData10, inData11, inData12)
write.table(outData, "Total_XYZ_Before_Projection.txt", sep=",", col.names=FALSE, row.names=FALSE)

Web Map Application in ArcGIS online

Create a public account in ArcGIS online website

R - Split a String at 2 characters into 3 columns

Before this please refer to the post "Export list of Feature datasets and classes from a geodatabase to CSV"

Here you will observe that each line has 2 characters. One is " " and second one is "\". We need to bring this file into this format: Geodatabase Name, Feature Dataset Name, Feature Class Name. 

To process an above mentioned text file, this R code has been prepared in R Studio.

# read input file

input_file <- "I:/GDBs/GDB2csv.txt"

file_con <- file(input_file, "r")

input_data <- readLines(file_con)

close(file_con)

# read each line and reformat

output_data <- NULL

for (eachLine in 1:length(input_data)) {

  out_str1 <- unlist(strsplit(input_data[eachLine], "\\\\"))

  if (length(out_str1) == 3) {

  out_str2 <- unlist(strsplit(out_str1[3], ".gdb "))

  output_tmp <- data.frame(Geodatabase_name = out_str2[1],

                           Feature_dataset_name = " ",

                           Feature_Class_name = out_str2[2],

                           stringsAsFactors = FALSE)

  

  } else if (length(out_str1 == 4)) {

    out_str2 <- unlist(strsplit(out_str1[4], " "))

    output_tmp <- data.frame(Geodatabase_name = out_str1[3],

                             Feature_dataset_name = out_str2[1],

                             Feature_Class_name = out_str2[2],

                             stringsAsFactors = FALSE)

    

  } else {

    message("check input!")

  }

  

  # update output

  output_data <- rbind(output_data, output_tmp)  

}

# write output

write.table(output_data, file = "I:/GDBs/GDB2csv_Output.txt", sep = ",", 

            quote = FALSE, row.names = FALSE)

The output text file is having 3 columns: Geodatabase Name, Feature Dataset Name, Feature Class Name. 

The purpose of this exercise is to maintain the multiple geodatabases accessed by users from several locations without an ArcSDE server.  It helps an user to determine which feature dataset or feature classes has been added to particular geodatabase from other users. The limitations of this program is that  an user will not be able to check the last updated version of any feature classes. Infact, it won't be necessary because all the users from multiple location will going to access same geodatabase on a server.

Python (ArcPy) - Export list of Feature datasets and classes from a geodatabase to CSV

These are 35 geodatabases and each geodatabases have several feature datasets and feature classes. Each feature datasets has several feature classes as well.

Open an ArcPy window in ArcGIS and type the below mentioned python code.

import arcpy

dir = r'I:\GDBs'

arcpy.env.workspace = dir

gdbList = arcpy.ListWorkspaces('*','FileGDB')

for gdb in gdbList:

    arcpy.env.workspace = gdb               #--change working directory to each GDB in list

    datasetList = arcpy.ListDatasets('*','Feature')     #--make a list of all (if any) feature datasets that exist in current GDB

    fcList = arcpy.ListFeatureClasses()         #--make a list of all feature in current GDB (root)

    for fc in fcList:

        print arcpy.env.workspace,fc            #--print directory,fc name

    for dataset in datasetList:

        arcpy.env.workspace = dataset   #--change working directory to each dataset (if any) in list

        fcList = arcpy.ListFeatureClasses()     #--make a list of all feature in current GDB (current dataset)

        for fc in fcList:

            print arcpy.env.workspace,fc        #--print directory,fc name

        arcpy.env.workspace = gdb

It will give you the output like this:

Copy the output in a text pad abd saved as GDB2csv.txt. I have only copied 30 lines out of approximately 1000 lines present in the textpad. Each line represent one feature class.

Here you will observe that each line has 2 characters. One is " " and second one is "\". We need to bring this file into this format: Geodatabase Name, Feature Dataset name, Feature Class Name. 

After this please refer to the post "Split a String at 2 characters into 3 columns"