To get to the point - Junge-Wissenschaft
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Paper 06 / 2021 doi: 10.7795/320.202106 Verlag: Physikalisch- Technische Bundesanstalt JungforscherInnen publizieren online | peer reviewed | original Mathematik & Informatik To get to DER JUNGFORSCHER the point Neural Network application to key-point detection in radiographs Physicians have to locate so called key-points e. g. for surgical procedures. Up to now, this was always done manually. In order Constantin Tilman Schott to automate this process, innovative software was developed (2003) that uses artificial intelligence (AI) combining a clipping- Paul-Gerhardt-Schule window approach with the newly developed prediction shifting. Dassel The program can predict the key-points with a high degree of Eingang der Arbeit: accuracy–making the AI as precise as a physician. 6.10.2019 Arbeit angenommen: 6.12.2019
Mathematik | Seite 2 hips etc.), specific structures have to be marked with precise points. This method is used in almost all orthopaedic or surgical X-ray analyses. The angles of these points in relation to each other and their distances from each other help the physician make a specific diagnosis. This process is known as predictive analytical key-point detection. The goal of this project is to use CNNs to automate this process in order to enable fully autonomous analysis in other areas of X-ray diagnostics over the long term. This involves applying various CNN structures (existing ones as well as self-developed ones) to the radiological analysis (X-ray image analysis) and drawing comparisons between them. To get to There are already AI methods that include localization (using facial the point recognition, for example) [14]. This paper discusses the applicability of these methods to the issue at hand here. In addition, new methods are being developed that allow for more accurate key-point detection. Neural Network application to key-point detection in radiographs In this paper the cephalometry (measurement of the cranium) is examined as a medical subfield, in which key-point detection is the mainly used 1. Introduction and most important analysis technique. Cephalometry is used in areas such as The results of the X-ray image analysis made it possible to achieve an extreme orthodontic diagnostics and therapy provide the basis for medical diagnostics increase in efficiency and accuracy planning, aesthetic surgery planning and treatment planning. Up to now, in automated detection of structures and, sometimes, in post-traumatic these analyses are performed manually, in recent years [3, 9]. This can be seen reconstructive surgery planning. making them a drain on time and exemplary at the MNIST database [6] human resources, and an expensive task which is used frequently to evaluate the As a starting point, methods are in the specialized medical field of X-ray quality of image recognition methods developed in this paper to automatically image analysis. In light of these facts, using the example of handwritten digit locate the so-called Sella point, that was this paper will explore the option of a recognition. chosen for its great importance for the standardized, computer-based analysis analysis. It is used as a datum point for of X-ray images. These CNNs have already started being almost all cephalometric analyses and used to deal with classification tasks in is the center of the Sella turcica (Latin There are a variety of automated, X-ray diagnostics. For example, you can for “Turkish seat”). The Sella turcica computer-aided approaches used to analyze a mammography for cancer foci is located in the centre of the cranium extract information from images – [13]. There are already many examples of base and, as such, is a nearly constant finding “features” in images. The CNNs for chest x-rays, even commercial datum point for these analyses, best results here are achieved by products (oxipit). regardless of factors such as growth- artificial intelligence in the form of related or traumatic changes to the bony convolutional neural networks (CNNs) In addition to these classification tasks, viscerocranium. [2]. Among other benefits, CNNs have in many X-ray image analyses (cranium, doi: 10.7795/320.202106
JUNGE wissenschaft 06 15 / 21 18 | Seite 3 On top of that, the following basic AI methods were used in this work in the experiments: Sella point In order to reduce overfitting dropout was used. This involves randomly deactivating a fixed percentage of neurons in every learning step in the respective layer it is being applied to and optimizing the network without these neurons [12]. This distributes the learning process across more neurons and increases the probability of detecting relevant patterns and structures, Fig. 1: Sella turcica in Fig. 2: Sella turcica with and thus counteracts overfitting. The lateral cephalogram Sella point drawn in training loss converges more slowly as a consequence of dropout, since it is not the entire network that is learning After successfully automating this of the Sella turcica also varies (Fig. 3). simultaneously. key-point-analysis of the Sella point, The purpose of automated analysis is to the developed method should be yield a reliable result, comparable to a Furthermore, data-augmentation was transferred on other important points professional medical analysis, in this applied: of the cephalometric X-ray analysis. complex initial situation. In the lateral cephalogram analysis, the 2. Preliminary considerations 2.2 Basic AI methodology points to be found cumulate heavily. and methodology As a result, a prediction of the middle There are three basic possible app- of the point cloud that is independent 2.1 Problem statement lication areas for the CNNs used in this of the input might be enough for a low paper: deviation with respect to the individual As mentioned above, the Sella turcica lateral cephalogram image (Fig. 4). is used on an X-ray image of the side 1. Classification of images (e. g. binary of the cranium (lateral cephalogram) classification → output between In the actual application example, as an application example. On the 0 and 1) the original images (without data two-dimensional projection of the augmentation) are shifted by a random cranium using a lateral cephalogram, 2. Key-point detection (→ output of a value while making sure, that the Sella the Sella turcica can be identified as an coordinate of the point(s)) itself stays within the image boundaries oval opening upward. The Sella point (Fig. 5). (S-point) “is defined as the (geometric) 3. Image segmentation (→ output of an center of the bony crypts of the Sella image on which, for example, the As a result, the network is forced to turcica” [15] (Fig. 1 and 2). detected features are drawn) learn only the truly relevant correlations between inputs. If the network The boundaries of the structure must be In this work, these three application optimized to the augmented input identified individually from one lateral options (sometimes in combination) are is then tested on normal application cephalogram to the next. The shape used to locate the Sella points. examples without augmentation, the Fig. 3: Various Sella turcica structures with Sella point marked doi: 10.7795/320.202106
Mathematik | Seite 4 Fig. 4: Distribution of Sella points Fig. 5: Distribution of Sella points (without data augmentation) (with data augmentation) network should detect them with a and optimization function (with calculated data were analyzed and higher degree of accuracy compared to learning rate etc.). For all nets the Adam graphically depicted using the Python a net without image augmentation. [8] optimization function and the ReLu Matplot library [18]. activation function [4, 17] turned out to On top of that, histogram equalization be optimal. All the programs used for this paper was utilized. Lateral cephalogram are stored as a GitHub project at https:// images sometimes have significantly 2.3 Tools github.com/tinotil/DeepLearning _ different contrast values. These values FRS_JuFo_2019. can be offset by a histogram equalization Programming was performed on a [5] (Fig. 6). computer with Intel i7-8700-CPU, 2.4 Data 32 GB DDR4-RAM and a Nvidia For all the nets the hyperparameters Geforce GTX 6000Ti with 6 GB RAM 420 anonymized lateral cephalograms used were optimized in a random and with an Ubuntu 18.04 LTS of the side of the cranium (FRS) search [19]. operating system. The programming analyzed by medical specialists are the environment consisted of a Jupyter basis of all the following analyses. The These hyperparameters include the size notebook [7] with browser-based server- pictures themselves where taken from a of the batches the training dataset was client architecture, where the server private orthodontic doctor’s office. The split into, the number of convolutional used the local host environment via a optimal points for each cephalogram layers (and their number of kernels and loopback. The programming language are the geometric mean of the points their respective sizes), the max-pooling used was “Python” combined with of two doctors with an experience in layer kernel-sizes, the amount of the “TensorFlow” library with GPU this kind of x-ray image analysis of over dropout, the size of the fully-connected support and the “Keras” deep learning 25 years. layers on top (if used), the loss function library with TensorFlow backend. The Fig. 6: Comparison of a Sella image with weak contrast before (left) and after a histogram equalization (right) and the associated pixel intensity histograms doi: 10.7795/320.202106
JUNGE wissenschaft 06 15 / 21 18 | Seite 5 Fig. 7: Example structure of the U-Net (according to Ronneberger et al. [10]) 300 of the images were part of the enable the detection of multiple faces at interpreted as the predicted S-point. training data set, 100 made up the once [14]. validation data set and 20 made up the As shown in Figure 7, a multi-channel test data set. While this assignment In medical applications comparable feature map is created in the left half remains the same for a learning cycle approaches are also already used in (CNN), which then gets turned back (first to last epoch), the images are the Biomedical Image Segmentation into an output image through up- randomly re-assigned to these data and feature Localization with e. g. sampling and other convolutions in the sets (cross-validation) before each so-called U-Nets [10]. Such a U-Net right half. subsequent learning cycle (weights structure was used as a first approach. reset). In this structure, another part with 3.2 Key-point detection up-convolutions (expansive path) was network using entire lateral The original image has a size of attached to the classic CNN (contracting cephalogram 993 × 1123 pixel with 150 dpi. path). For finding the S-point, the lateral cephalograms were used as the input 3.2.1 Pre-trained key-point detection The pieces of data listed below each and a corresponding image matrix network represent the mean values of five with the maximum activation at the learning cycles (5-fold CV). optimal point was used as the output. In As a second approach a CNN was addition, the pixels around the optimal used and its output feature maps are 3. Evolution of approaches point were activated to the effect of interpreted by a DenseNet, which itself a statistical standard distribution in outputs a 2 × 1 vector - the predicted 3.1 Image segmentation: order to enable gradual convergence S-point coordinate. Heatmap approach to the optimal point. The optimal scaling factor of this Gaussian was The deep learning library Keras offers A heatmap method is used as the first experimentally determined and then different pre-trained CNNs whose approach for automated localization of fixed for all further experiments. This weights have already been optimized to the Sella point on a lateral cephalogram. creates what is known as the heatmap certain applications, and are therefore This process is based on facial for training. The point of maximum able to detect basic structures reliably. recognition CNNs, where heatmaps activation of the output-heatmap gets During an initial test, a pre-trained doi: 10.7795/320.202106
Mathematik | Seite 6 VGG16 network [11] was used with a Dropout was utilized as needed evaluated on two levels using a clipping two-layer DenseNet. increasing from 0.2 up to 0.7. window approach. First, a classification CNN classifies selected partial images The coordinates of the points are 3.2.2 Custom key-point according to the categories “Sella” standardized for all key-point detection network or “not Sella”. In the second step, detections. In doing so, each coordinate a key-point detection network sets value was divided by the width of the In the next step, the pretrained VGG16 the S-point on the classified images respective image edge (x-coordinate/ network was replaced by a custom of a Sella turcica. width of the image, y-coordinate/ height trained, smaller network with only 4 of the image). As a result, the expected convolutional layers and two dense Since only a section of the lateral outputs were limited to a range between layers on top. cephalogram image is input (partial 0 and 1. Here, only the DenseNet and image size 150 × 150 pixels) into a the top two layers of the VGG16 network In this example data augmentation network, down-sampling no longer were trained. This was done to prevent was used extensively varying between needs to be applied to the input lateral overwriting weights of the lower layers a random shift over 3/4 or 1/2 of the cephalogram image. As a result, a lateral that had already been optimized and output range. cephalogram image that is 750 × 750 to reduce the training time. The lateral pixels can be used as an initial input, cephalogram images were used as input In order to enable a learning process thereby increasing the information data and the standardized coordinates the pictures had to be down-sampled density. of the S-points were used as output. As to an input size of 150 × 150 pixels from the VGG16 network uses RGB images as an original size of 993 × 1123 pixels. A 3.3.1 Classification network input the grayscale channel values were larger input cannot be implemented in duplicated to generate a three-channel this key-point detection network due to A simple CNN with only 3 convolutional picture. These input images were also hardware limitations. layers and two interpreting Dense layers rescaled using PIL (Python Image was used as a classification network. Library) in order to fit the VGG16 input 3.3 Clipping window approach In this process, an output of 0 was dimensions. assigned for Sella and 1 for non-Sella. An improved approach for precisely Therefore, a sigmoid function was detecting the S-point is based on the used as an activation function of the process of Ciresan et al. [1]. The image is output layer. Since all outputs greater Fig. 8: Example of generating partial images Fig. 9: Deviations of predictions from the expected based on the Sella points that have already values in mm for 20 test Sella images, depending been identified in the entire lateral on the position of the ground truth point on cephalogram (with 10 points for the Sella partial image (image center: purposes of simplification) 75|75 pixels) doi: 10.7795/320.202106
JUNGE wissenschaft 06 15 / 21 18 | Seite 7 than 0.5 can be interpreted as 1 and all only the image with the highest degree partial image was recentered, with the those less than 0.5 can be interpreted of equivalence to a Sella is ultimately previously predicted Sella point as the as 0 in the binary classification of selected. new centre. Only the second prediction this case example, the model is not as then is the final prediction of the susceptible to overfitting. For learning If none of the partial images output network. The network used is the same purposes, the corresponding Sella using the described search method yield as in 3.3.2. image was generated from each lateral an equivalence of at least 99.9 %, a simple cephalogram with the set Sella point as overlapping sliding window process As a second measure the coordinate was the middle point. The non-Sella images (fixed offset in x- and y-directions) is split into two separate scalars for the were able to be generated by randomly used. x and the y coordinate that are being selecting partial images from the X-ray predicted by separate networks. images. The edges of the non-Sella 3.3.2 Key-point detection network images adhered to a minimum distance using Sella partial image 4. Results of 75 pixels from the set Sella point. Again, a key-point detection network 4.1 Image segmentation: To ensure practical use of the network, a outputs the exact coordinate of the Heatmap approach method must be developed that divides Sella point. This time, however, it is the entire input lateral cephalogram not operating on the entire lateral Learning is not possible for this kind into partial images that are then divided cephalogram, but only on the partial of network in this application example. into categories of “Sella” and “non- image containing the Sella structure, The U-Net learns to output a completely Sella” by the classification network. that has been positively classified by the non-activated output, by adapting to There are a variety of search patterns classification net. the desired output. Similar results can that can be used to accomplish this. A be seen when using different heatmap square with a side length that is 1/5 of Image augmentation, an even generation methods (other than the total image height is used as the size distribution of optimal points in the U-Nets). of these images. output area, is absolutely necessary for the training set, since the images were The heatmap approach is a quantitative The a priori knowledge of the created for the training data set based key-point detection. This means that the accumulation of S-points on the lateral on the optimal point. Without image focus is on detecting multiple structures cephalogram images (Fig. 4) can be augmentation, the S-point would always (e. g. faces) simultaneously, during used to generate pictures of a high be in the center of the image. which the accuracy of an individual probability of containing the Sella. prediction is less relevant. During X-ray 3.3.3 Key-point detection network image key-point detection, each point First, all the manually set Sella points using Sella partial image with only needs to be detected and located that have already been analyzed once are prediction shifting once, but with very high accuracy, which used as centres of the generated images, is not possible using this approach. which are then fed into the classification A consideration of the average error, network as input to be classified (Fig. 8). depending on the position of the Sella Therefore, qualitative key-point on the partial image (based on the detection methods must be developed As already mentioned, the classification optimal points), results in the following and tested. network uses a sigmoid function as an distribution (Fig. 9). activation function of the output layer. 4.2 Key-point detection Unlike functions such as the heaviside The prediction becomes more accurate network using entire lateral step function, the advantage here is the more centrally the Sella is located. cephalogram that differences in equivalence between Between the edge areas and the the input and the learned patterns are center, the deviations can be seen to 4.2.1 Pre-trained key-point detection depicted. Despite this, a clear distinction be reduced by more than half. This network (0 or 1) is favored through a larger can be correlated with the higher gradient (major change in each learning information density of the central areas The quality of this approach can be step) in the value range between 0 and 1. (see “valid padding”), as well as better seen in Fig. 10, where the deviation of Therefore, in the clipping window visibility of the relevant structures. the validation data set from that of the approach, the network can assess the To make use of this fact, the same training data set is shown. The diagram degree of similarity to the Sella that net was used to make two separate shows clear overfitting. The deviation of the input image depicts. Accordingly, prediction. Between the predictions the the training data set converges to zero, doi: 10.7795/320.202106
Mathematik | Seite 8 Fig. 10: Training and validation loss over Fig. 11: Training and validation loss over epochs (pre-trained key-point detection epochs (pre-trained key-point detection network; no dropout) network; dropout of 0.2) although the deviation of the validation for the task at hand than the complex A dropout of 0.2 prevents overfitting data set fluctuates significantly and one used previously. here as well, and allows the model does not converge. This means that the to achieve an even lower average test network can set the learned points in 4.2.2 Custom key-point detection deviation of 2.8 % of the image size the training data set very precisely but network (4.2 mm). fails at new, unseen tasks. An average deviation of 23 % of the image size The learning progress of the custom net The average deviation of predictions (34.5 mm) in the test data shows similar can be seen in (Fig. 12). of the test data set (3 % of image size, discrepancies as the validation data set. 4.5 mm) differs from the average This simplified and more specialized deviation of predictions of the validation Using a dropout of 0.2 (deactivation model learns more quickly because data set (2 % of image size, 3 mm). of every fifth neuron), it was possible fewer parameters have to be optimized to make noticeable improvements to (30 seconds for 30 epochs instead of The specific Data-Augmentation used the overfitting problem (Fig. 11). The 2.30 minutes for 30 epochs with the on the custom network did not improve predictions of the test data set were pre-trained model). Here, an average the results. made with an average deviation of 11% deviation of just 3.2 % of the image of the image size (16.5 mm) using this size (4.8 mm) can be identified with the By randomly shifting the points onto an new model. test data set. Using additional dropout, area that is 3/4 of the image area, only overfitting can also be minimized here. a test deviation of 8 % of the image size Despite the dropout, we also have a relatively high fluctuation of validation loss in this adjusted model. If the dropout is increased even further, the deviation can also be further reduced with a very high dropout up to a certain point. At a dropout of 0.5 (50 % of all neurons are deactivated in every learning step), the deviation was able to be reduced to 7 % of the image size (10.5 mm), and to 6 % of the image size (9 mm) with a dropout of 0.7. Such a positive reaction to high dropout Fig. 12: Training and validation loss over epochs (custom speaks to the fact that a simpler network key-point detection network; no dropout) would be able to achieve better results doi: 10.7795/320.202106
JUNGE wissenschaft 06 15 / 21 18 | Seite 9 Fig. 13: Waterfall plot of prediction errors for 20 Sella partial images (test data set) (12 mm) could be achieved, and 7 % Using the clipping window search 4.3.3 Key-point detection network (10.5 mm) at 1/2 the image area. algorithm the Sella structure was using Sella partial image with identified every time in each of the 20 prediction shifting 4.3 Clipping window approach test images and an image of a complete Sella was outputted each time. Fig. 13 compares the approach of 4.2.3 4.3.1 Classification network (left) and of 4.3.3 (right). 4.3.2 Key-point detection network After adjusting the network structure using Sella partial image Both networks described 4.3.2 and 4.3.3 and the dropout, the network achieves an are applied to the Sella partial images accuracy of 98.8 % correct classification After a total of 60 training epochs, an created using the classification network for lateral cephalogram partial images average deviation of the test data set of described in 4.3.1 from the whole X-ray after 150 epochs. 2.5 % of the partial image size (0.8 mm) images. was achieved. While this approach Further improvement to 99.5 % correct exhibits a low average deviation, The classification process (4.3.1) classification for lateral cephalogram the scattering of the losses is high consequently extracts Sella partial partial images was achieved using (at maximum 10 %, 3.2 mm) with a images with a size of 150 × 150 pixels histogram equalization. standard deviation of 1.2 mm. from the entire input lateral cephalogram which itself has a Fig. 14: Deviation of predictions with respect to the points set by specialists: Left: Pre-trained key-point detection network (3.2.1) Middle: Custom key-point detection network (3.2.2) Right: Clipping window approach with prediction shifting (3.3) Red circle: Average Sella size doi: 10.7795/320.202106
Mathematik | Seite 10 standardized size of 750 × 750 pixels. The prediction shifting, as tested on the test number of activation cumulations exact middle point of the Sella now can data set, is clearly the most precise. on the heatmap) is very low (singular be determined on these partial images cumulation). Therefore, learning is not by the key-point detection net (4.3.3) 4.5 Application on other points possible for this kind of network in this with an average deviation of 1.8 % and application example. The U-Net learns maximum 5 % of the partial image size. To test the transferability of the to output a completely non-activated In order to make this approach developed method, it was tested on the output, by adapting to the desired comparable to the prior ones the error incisal point. This is another important output. Due to the low stimulus density, should not be correlated to the Sella point to be placed on the foremost tip of the parts of the desired heatmap partial image but to the entire lateral the first incisor (Fig. 15, 16). that are non-activated are too large. cephalogram (as in 4.1 and 4.2), leading These problems could be addressed by to an average deviation of 0.36 % The mean deviation of 0,65 mm and enlarging the distribution radius of of the image size and a maximum maximum deviation of 1,4 mm is very activations, although this inevitably deviation of 1 % of the image size. comparable to the results seen on the would make the output less specific. The Based on an original lateral original application on the Sella point. same problem also applies to possible cephalogram as it is presented to the heatmap approaches without using a doctor, this corresponds to an average 5. Discussion U-Net structure. However, this kind of deviation of 0.6 mm and a maximum method could help in other radiological deviation of 1.5 mm. In this project, different methods tasks, such as detecting carious areas, were developed and compared for since this involves quantitative key- 4.4 Comparison of approaches automatically carrying out key-point point detection. detection on X-ray images using Finally, if the three processes examined neural networks. The research used A key-point detection network was in this paper are compared based on the the example of the Sella point on used in the next step. The reduction absolute deviations for the respective lateral cephalograms of the side of the of complexity of the output allowed coordinates of the Sella point (Fig. 14), cranium. Three different approaches predictions that achieved an average the increase in accuracy can be seen were compared. deviation of just 4.2 mm with the clearly, as has been shown in the course redesigned and custom network. A of this project. The average deviation A heatmap approach, as is used in key-point detection network is useful of 6 % (9 mm) in the optimized first facial recognition, was introduced in for finding individual, specific points. approach was reduced to 0.36 % of the the first step. This network structure is Due to the limitations of the available image size (0,6 mm) in the third process. particularly well-suited for detecting hardware, the size of the lateral multiple points at the same time, but cephalogram images needed to be The following three diagrams (Fig. 14) less so for precisely setting individual reduced from the original 993 × 1123 to shows the deviation of predictions key points. 150 × 150 pixels. of the respective networks in the x- and y-direction. Here, the custom, During X-ray image key-point detec- Further important localization combined network from 3.3 with tion, the stimulus density (i.e. the information is lost due to the above Incisal point Fig. 15: Incisor in lateral cephalogram Fig. 16: Incisor with Incisal point drawn in doi: 10.7795/320.202106
JUNGE wissenschaft 06 15 / 21 18 | Seite 11 mentioned max pooling and valid Sella point on which these methods This ability is especially important in padding. However, these methods were tested was able to be located with localizing the Sella point, since this is were necessary in this application in an average deviation of 0.6 mm. an imaginary point. This means that order for the CNN error to converge the point is not set on a real structure, while keeping the number of optimized After a reliable way to automatically but rather has to be constructed based parameters within the bounds set by the detect the S-point has been developed, on the surrounding structure of the hardware. this method can be applied to other Sella turcica. This makes localization relevant points of cephalometric X-ray difficult and demonstrates the ability of This sets a upper limit on the possible analysis. the model to not only detect features, accuracy of key-point localization. but also to correlate different detected This type of network has shown itself Unlike some algorithms or programs features to each other. to be suitable for finding points in this which are not capable of learning, a research, but it exhibits significant neural network structure is not tied to The application to the incisal point has precision problems in prediction. the original application area. Instead, furthermore shown a certain degree of it can be transferred to other problems transferability, in particular to a non- Two neural networks that build on each using the same solution approach with imaginary point which is to be placed other were used as a third approach. little to no change in accuracy. on an actual depicted structure. First, a classification CNN extracts the partial image from the entire image This transferability of this approach was The actual transferring of this method containing the Sella. Only once this proven by testing it on the incisal point. to other points can be executed in partial image is extracted does a key- The deviation only differs by less than further research. point detection network set the precise 0,01 mm from the results of the Sella S-point on it. This limits the influence point which shows the applicability of Outside of lateral cephalogram analysis, of the loss of localization information the method to different points. there are a few other X-ray image of the key-point detection network on evaluations that have so far applied final prediction. In the context of a manual evaluation by manual key-point detection. This a specialist, Segner and Hasund write the includes all orthopaedic, bone-related This approach is also based on the way following about the reproducibility of malpositions, i.e. malpositions of the a doctor works, first by searching for manual point localizations: “It generally hip, knee or spine. The transferability of the most likely search area for the Sella will not be possible to reproduce the the process developed here can also be structure, and then (after detecting the measured values of the first evaluation tested in this field. Sella) approximating the S-point. with an accuracy of 0.1 or 0.5 mm, no matter whether the evaluation was done 6. Outlook Lastly the method of prediction shifting (by a doctor) digitally or by hand” [16]. was used based on the fact that a more Based on these results, the next goal in central position of the relevant structure The deviation of the neural network this process is the automated evaluation leads to a more accurate prediction. from the points set by professionals that of other points based on the described was determined in this paper is thus method of the clipping window with It could be speculated that this is comparable to the deviation that occurs prediction shifting. To do so, new data correlated with the higher information between two medical evaluations. The sets are required that must be created by density of the central areas (see “valid developed method thus enables practical hand. padding”) as well as better visibility of results for radiologic evaluation to be the relevant structures in the centre. achieved. The long-term goal here is being able to perform completely autonomous Using the clipping window and However, the existing results thus far X-ray image evaluation (possibly with prediction shifting, the divide and only pertain to the example application diagnosis) in cephalometry based on conquer paradigm was successfully of the methods on two points. key-point detections by applying this applied. method. In localization of other important The goal of developing a workable points on a lateral cephalogram as well, Here, only the angles and distances procedure for predictive analytic key- certain features have to be detected and between the individual points need point detection was achieved using correlated in order to approximate the to be calculated in the last step. The the clipping window approach and the respective position. appropriate software already exists for prediction shifting developed here. The this purely algebraic task. However, doi: 10.7795/320.202106
Mathematik | Seite 12 at the moment, it only produces This project succeeded in identifying calculations based on points set by the Sella point with an average deviation hand. to the manually set points of less than 1 % of the image size and an absolute An opportunity for the direct deviation of 0.6 mm. This is within the implementation of the system in an spread range for specialist evaluation. application environment is thereby given. Using the method developed by In the next step, transferring the combining a clipping window process, methods to similar X-ray image a classification network, two key-point evaluations (hip, knee, or spine X-ray detection networks and prediction images) is to be tested. shifting, the method has already been successfully tested on an additional Any real-world implementation of this point of cephalometric analysis. The kind of automation will certainly still transferability has been proven. (currently) need the supervision and review of a specialist, but the evaluation Acknowledgements can be accelerated and standardized. That is an advantage for both the doctor I would like to express my thanks for and for the patient. This application area the support from Dr. Carsten Winkler is an example of a beneficial human- (mathematics and physics teacher, PGS machine collaboration. Dassel) for the critical examination of my paper as well as Drs. Angrit In following projects, image and Thomas Schott — my parents preprocessing could be examined in — for providing the anonymized greater detail. In this paper, histogram lateral cephalograms and the manual, equalization was applied to compensate professional setting of the Sella points. for contrast differences. Other methods include edge detection or thresholding, for example. In both methods, irrelevant data can be discarded before beginning training, which enables faster and more reliable detection by the network. This can reduce the necessary network depths and, as a result, the calculation effort as well. On the other hand, preprocessing images too heavily can destroy important structures on some images. Individualized preprocessing tailored to each image increases the effort needed for learning — a compromise needs to be found. 7. Summary The goal of this research was to develop methods for automated key-point detection on X-ray images, using the example of Sella points within lateral cephalogram of the side of the cranium with the aid of artificial intelligence, specifically the use of convolutional neural networks. doi: 10.7795/320.202106
JUNGE wissenschaft 06 15 / 21 18 | Seite 13 Sources and bibliography [12] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout. A [1] Ciresan, D.C., Gambardella, L.M., Giusti, A., Simple Way to Prevent Neural Networks from Schmidhuber, J.: Deep neural net-works Overfitting. In: Journal of Machine Learning segment neuronal membranes in electron Research. Volume 15, No. 2. Department of microscopy images. In: NIPS. pp. 2852–2860 Computer Science University of Toronto, (2012) p. 1929–1958 (2014) [2] Ciresan, D., Meier, U. ,Schmidhuber, J.: Multi- [13] Platania, R., Shams, S., Yang, S., Zhang, J., column deep neural networks for image Lee, K., Park, S: Automated Breast Cancer classification. In: Institute of Electrical and Diagnosis Using Deep Learning and Region of Electronics Engineers (IEEE), p. 3642–3649. Interest Detection (BC-DROID) In: Proceedings (2012) of ACM-BCB’17, August 20-23, 2017, Boston [3] Girshick, R., Donahue, J., Darrell, T., Malik, J.: [14] Do Nhu, T., Kim, S.H. , Yang, H.J., Rich feature hierarchies for accurate object Lee, G.-S.: Face Tracking with Convolutional detection and semantic segmentation. Neural Network Heat-Map, 2018, Conference: In: Proceedings of the IEEE, Conference on The 2nd International Conference on Machine Computer Vision and Pattern Recognition Learning and Soft Computing (ICMLSC 2018), (CVPR) (2014) At Phu Quoc, Vietnam [4] Hahnloser, R., Seung, H.S.: Permitted and [15] Segner, D., Hasund A.: Individualisierte Forbidden Sets in Symmetric Threshold-Linear Kephalometrie, Dietmar Segner Verlag, Networks In: Neural Computation archive, Hamburg, 1998, 3rd Edition, p. 14 Volume 15, Issue 3, pp. 621–638 (2003) [16] Segner, D., Hasund A.: Individualisierte [5] Abdullah-Al-Wadud M., Hasanul Kabir Md., Kephalometrie, Dietmar Segner Verlag, Ali Akber Dewan M., Chae O.: A Dynamic Hamburg, 1998, 3rd Edition, p. 42 Histogram Equalization for Image Contrast [17] Leshno M., Lin V. Ya., Pinkus A., Enhancement, In: IEEE Transactions on Schocken Sh.: Multilayer feedforward Consumer Electronics (Volume: 53 , Issue: 2 , networks with a nonpolynomial activation May 2007 ) function can approximate any function, [6] http://yann.lecun.com/exdb/mnist/ (retrieved Neural Networks. 6 (6), pp. 861–867, Jan 1993 on 2018-11-12, 17:14) [18] Hunter, J.D.: Matplotlib: A 2D graphics [7] https://jupyter.org/ environment, Computing in Science & (retrieved on 2021-07-1, 15:00) Engineering, Volume 9, p. 90–95, 2007 [8] Kingma, D., Ba, J.: Adam: A Method for [19] Bergstra J., Bengio Y.: Random Search for Stochastic Optimization In: ICLR (2015) Hyper-Parameter Optimization In: Journal of [9] Krizhevsky, A., Sutskever, I., Hinton, G.E.: Machine Learning Research. 13, pp. 281–305, Imagenet classification with deep 2012 convolutional neural networks, 2012 In: NIPS'12 Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, p. 1097–1105 (2012) [10] Ronneberger, O., Fischer, P. ,Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, Vol.9351: p. 234–241 (2015) [11] Simonyan, K. , Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition In: ICLR 2015 (2015) doi: 10.7795/320.202106
Mathematik | Seite 14 Publiziere auch Du hier! Forschungsarbeiten von Schüler/Inne/n und Student/Inn/en In der Jungen Wissenschaft wer- den Forschungsarbeiten von Schüler- Innen, die selbstständig, z. B. in einer Schule oder einem Schülerforschungs zentrum, durch geführt wurden, ver- Wie geht es nach dem öffentlicht. Die Arbeiten können auf Einreichen weiter? Deutsch oder Englisch geschrieben sein. Die Chefredakteurin sucht einen ge- Wer kann einreichen? eigneten Fachgutachter, der die in- haltliche Richtigkeit der eingereichten SchülerInnen, AbiturientInnen und Arbeit überprüft und eine Empfehlung Studierende ohne Abschluss, die nicht ausspricht, ob sie veröffentlicht wer- älter als 23 Jahre sind. den kann (Peer-Review-Verfahren). Das Gutachten wird den Euch, den AutorIn- Was musst Du beim nen zugeschickt und Du erhältst gege- Einreichen beachten? benenfalls die Möglichkeit, Hinweise des Fachgutachters einzuarbeiten. Lies die Richtlinien für Beiträge. Sie ent- halten Hinweise, wie Deine Arbeit auf- Die Erfahrung zeigt, dass Arbeiten, die gebaut sein soll, wie lang sie sein darf, z. B. im Rahmen eines Wettbewerbs wie wie die Bilder einzureichen sind und Jugend forscht die Endrunde erreicht welche weiteren Informationen wir be- haben, die besten Chancen haben, die- nötigen. Solltest Du Fragen haben, dann ses Peer-Review-Verfahren zu bestehen. wende Dich gern schon vor dem Ein- reichen an die Chefredakteurin Sabine Schließlich kommt die Arbeit in die Re- Walter. daktion, wird für das Layout vorberei- tet und als Open-Access-Beitrag veröf- Lade die Erstveröffentlichungserklärung fentlicht. herunter, drucke und fülle sie aus und unterschreibe sie. Was ist Dein Benefit? Dann sende Deine Arbeit und die Erst- Deine Forschungsarbeit ist nun in ei- veröffentlichungserklärung per Post an: ner Gutachterzeitschrift (Peer-Review- Die Junge Wissenschaft wird zusätzlich Journal) veröffentlicht worden, d. h. Du in wissenschaftlichen Datenbanken ge- Chefredaktion Junge Wissenschaft kannst die Veröffentlichung in Deine listet, d. h. Deine Arbeit kann von Ex- Dr.-Ing. Sabine Walter wissenschaftliche Literaturliste auf- perten gefunden und sogar zitiert wer- Paul-Ducros-Straße 7 nehmen. Deine Arbeit erhält als Open- den. Die Junge Wissenschaft wird Dich 30952 Ronnenberg Access-Veröffentlichung einen DOI durch den Gesamtprozess des Erstellens Tel: 05109 / 561508 (Data Object Identifier) und kann von einer wissenschaftlichen Arbeit beglei- Mail: sabine.walter@verlag- entsprechenden Suchmaschinen (z. B. ten – als gute Vorbereitung auf das, was jungewissenschaft.de BASE) gefunden werden. Du im Studium benötigst. doi: 10.7795/320.202106
JUNGE wissenschaft 06 15 / 21 18 | Seite 15 ein. Für die weitere Bearbeitung und Richtlinien die Umsetzung in das Layout der Jungen Wissenschaft ist ein Word- für Beiträge Dokument mit möglichst wenig Formatierung erforderlich. (Sollte dies Schwierigkeiten bereiten, setzen Für die meisten Autor/Inn/en ist dies die Sie sich bitte mit uns in Verbindung, damit wir gemeinsam eine Lösung erste wissenschaftliche Veröffentlichung. finden können.) Die Einhaltung der folgenden Richtlinien hilft allen – den Autor/innen/en und dem ■ Senden Sie mit dem Beitrag die Redaktionsteam Erstveröffentlichungserklärung ein. Diese beinhaltet im Wesentlichen, dass der Beitrag von dem/der angegebenen AutorIn stammt, Die Junge Wissenschaft veröffentlicht Sponsoren, mit vollständigem Namen keine Rechte Dritter verletzt Originalbeiträge junger AutorInnen bis angefügt werden. Für die Leser kann werden und noch nicht an anderer zum Alter von 23 Jahren. ein Glossar mit den wichtigsten Stelle veröffentlicht wurde (außer Fachausdrücken hilfreich sein. im Zusammenhang mit Jugend ■ Die Beiträge können auf Deutsch forscht oder einem vergleichbaren oder Englisch verfasst sein und ■ Bitte reichen Sie alle Bilder, Wettbewerb). Ebenfalls ist zu sollten nicht länger als 15 Seiten mit Grafiken und Tabellen nummeriert versichern, dass alle von Ihnen je 35 Zeilen sein. Hierbei sind Bilder, und zusätzlich als eigene verwendeten Bilder, Tabellen, Grafiken und Tabellen mitgezählt. Dateien ein. Bitte geben Sie bei Zeichnungen, Grafiken etc. von Anhänge werden nicht veröffentlicht. nicht selbst erstellten Bildern, Ihnen veröffentlicht werden dürfen, Deckblatt und Inhaltsverzeichnis Tabellen, Zeichnungen, Grafiken also keine Rechte Dritter durch die zählen nicht mit. etc. die genauen und korrekten Verwendung und Veröffentlichung Quellenangaben an (siehe auch verletzt werden. Entsprechendes ■ Formulieren Sie eine eingängige Erstveröffentlichungserklärung). Formular ist von der Homepage Überschrift, um bei der Leserschaft Senden Sie Ihre Bilder als www.junge-wissenschaft.ptb.de Interesse für Ihre Arbeit zu wecken, Originaldateien oder mit einer herunterzuladen, auszudrucken, sowie eine wissenschaftliche Auflösung von mindestens 300 dpi auszufüllen und dem gedruckten Überschrift. bei einer Größe von 10 ∙ 15 cm! Beitrag unterschrieben beizulegen. Bei Grafiken, die mit Excel erstellt ■ Formulieren Sie eine kurze, leicht wurden, reichen Sie bitte ebenfalls ■ Schließlich sind die genauen verständliche Zusammenfassung die Originaldatei mit ein. Anschriften der AutorInnen mit (maximal 400 Zeichen). Telefonnummer und E-Mail- ■ Vermeiden Sie aufwendige und lange Adresse sowie Geburtsdaten und ■ Die Beiträge sollen in der üblichen Zahlentabellen. Fotografien (Auflösung 300 dpi bei Form gegliedert sein, d. h. Einleitung, einer Bildgröße von mindestens Erläuterungen zur Durchführung ■ Formelzeichen nach DIN, ggf. 10 ∙ 15 cm) erforderlich. der Arbeit sowie evtl. Überwindung IUPAC oder IUPAP verwenden. von Schwierigkeiten, Ergebnisse, Gleichungen sind stets als ■ Neulingen im Publizieren werden Schlussfolgerungen, Diskussion, Größengleichungen zu schreiben. als Vorbilder andere Publikationen, Liste der zitierten Litera tur. In z. B. hier in der Jungen Wissenschaft, der Einleitung sollte die Idee zu ■ Die Literaturliste steht am Ende der empfohlen. der Arbeit beschrieben und die Arbeit. Alle Stellen erhalten eine Aufgabenstellung definiert werden. Nummer und werden in eckigen Außerdem sollte sie eine kurze Klammern zitiert (Beispiel: Wie Darstellung schon bekannter, in [12] dargestellt …). Fußnoten ähnlicher Lösungsversuche enthalten sieht das Layout nicht vor. (Stand der Literatur). Am Schluss des Beitrages kann ein Dank an ■ Reichen Sie Ihren Beitrag sowohl in Förderer der Arbeit, z. B. Lehrer und ausgedruckter Form als auch als PDF doi: 10.7795/320.202106
Mathematik | Seite 16 @ Impressum Impressum wissenschaft Junge Wissenschaft c/o Physikalisch-Technische Bundesanstalt (PTB) www.junge-wissenschaft.ptb.de Redaktion Dr. Sabine Walter, Chefredaktion Junge Wissenschaft Paul-Ducros-Str. 7 30952 Ronnenberg E-Mail: sabine.walter@verlag- jungewissenschaft.de Tel.: 05109 / 561 508 Verlag Dr. Dr. Jens Simon, Pressesprecher der PTB Bundesallee 100 38116 Braunschweig E-Mail: jens.simon@ptb.de Tel.: 0531 / 592 3006 (Sekretariat der PTB-Pressestelle) Design & Satz Sabine Siems Agentur „proviele werbung“ E-Mail: info@proviele-werbung.de Tel.: 05307 / 939 3350 doi: 10.7795/320.202105
You can also read